US20140331262A1 - Adaptive video server with virtual file system and methods for use therewith - Google Patents
Adaptive video server with virtual file system and methods for use therewith Download PDFInfo
- Publication number
- US20140331262A1 US20140331262A1 US14/284,195 US201414284195A US2014331262A1 US 20140331262 A1 US20140331262 A1 US 20140331262A1 US 201414284195 A US201414284195 A US 201414284195A US 2014331262 A1 US2014331262 A1 US 2014331262A1
- Authority
- US
- United States
- Prior art keywords
- video
- data
- encoded segments
- video programs
- context data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
- H04N21/47202—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for requesting content on demand, e.g. video on demand
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
- H04N21/234309—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4 or from Quicktime to Realvideo
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
- H04N21/262—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists
- H04N21/26258—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists for generating a list of items to be played back in a given order, e.g. playlist, or scheduling item distribution according to such list
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
- H04N21/266—Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
- H04N21/2662—Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/439—Processing of audio elementary streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
- H04N21/44209—Monitoring of downstream path of the transmission network originating from a server, e.g. bandwidth variations of a wireless network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
- H04N21/462—Content or additional data management, e.g. creating a master electronic program guide from data received from the Internet and a Head-end, controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
- H04N21/4621—Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/482—End-user interface for program selection
- H04N21/4825—End-user interface for program selection using a list of items to be played back in a given order, e.g. playlists
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
- H04N21/845—Structuring of content, e.g. decomposing content into time segments
- H04N21/8456—Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/85—Assembly of content; Generation of multimedia applications
- H04N21/854—Content authoring
- H04N21/8549—Creating video summaries, e.g. movie trailer
Definitions
- the present invention relates to servers used for streaming media applications including video on demand.
- each computer or Internet device can have its own Internet connection.
- each computer or Internet device includes a modem.
- an in-home wireless local area network may be used to provide Internet access and to communicate multimedia information to multiple devices within the home.
- each computer or Internet device includes a network card to access an IP gateway. The gateway provides the coupling to the Internet.
- broadcast video the Internet provides access to streaming video services. Instead of broadcasting all programming to every customer, each customer receives only those programs that they want, when they want them.
- HLS HTTP Live Streaming
- a video encoder takes a source video and encodes it at different bitrates intended for use in different network conditions (high or low capacity) and with different playback devices.
- a variant playlist is stored on the server's file system that describes one video that is available in multiple bitrates. For each available bitrate, a server-side tool takes the encoder's output file and breaks it into segments of equal duration (usually 10 seconds); these segments are stored on the server's file system. The segments may be optionally encrypted.
- a playlist For each available bitrate, a playlist is stored on the server's file system.
- the playlist specifies the properties (most importantly duration and URI) of the available segments.
- the encoded video, variant playlists and playlists must be prepared and available to the web server prior to streaming.
- the client initiates streaming by first requesting the variant playlist over HTTP.
- the client requests the first playlist listed in the variant playlist.
- the client requests the first segment in the playlist, then the second, and so on.
- the client will choose to switch to a playlist for a higher or lower bitrate, and proceed to request the video segments from the new playlist.
- the received segments are decoded and displayed by the client as one continuous video stream.
- HLS streaming is that the server is a standard web server. Beyond the application-specific tools needed for segmentation and playlist generation, the software and infrastructure comprises of standard and freely available components that are often already in place. The limitations and disadvantages of HLS and other conventional and traditional approaches will become apparent to one of ordinary skill in the art through comparison of such systems with the present invention.
- FIG. 1 presents a pictorial representation of a content distribution system that in accordance with an embodiment of the present invention.
- FIG. 2 presents a block diagram representation of streaming server 50 in accordance with an embodiment of the present invention.
- FIG. 3 presents a block diagram representation of playlist data 112 in accordance with an embodiment of the present invention.
- FIG. 4 presents a block diagram representation of variant playlist data 114 in accordance with an embodiment of the present invention.
- FIG. 5 presents a block diagram representation of an encoder 104 in accordance with an embodiment of the present invention.
- FIG. 6 presents a block diagram representation of an encoder 104 in accordance with another embodiment of the present invention.
- FIG. 7 presents a flowchart representation of a method in accordance with an embodiment of the present invention.
- FIG. 8 presents a flowchart representation of a method in accordance with an embodiment of the present invention.
- FIG. 9 presents a flowchart representation of a method in accordance with an embodiment of the present invention.
- FIG. 10 presents a flowchart representation of a method in accordance with an embodiment of the present invention.
- FIG. 1 presents a pictorial representation of a content distribution system in accordance with an embodiment of the present invention.
- a streaming video server 50 is capable of accessing and distributing content from one or more video sources 40 to a plurality of client devices such as a television 60 , smart phone, internet tablet or other personal media players 62 and 64 , handheld video player 66 , and personal computer 68 . While specific client devices are shown, video server 50 is capable of accessing and distributing content from one or more video sources 40 to other client devices that can receive and reproduce media content.
- client devices such as a television 60 , smart phone, internet tablet or other personal media players 62 and 64 , handheld video player 66 , and personal computer 68 . While specific client devices are shown, video server 50 is capable of accessing and distributing content from one or more video sources 40 to other client devices that can receive and reproduce media content.
- the streaming video server 50 uses an on-demand encoding process and a virtual file system.
- the streaming video server 50 allows many bitrates to be supported with no additional requirements for persistent storage.
- the streaming video server 50 accesses a virtual file system containing the files needed for streaming: variant playlist files, playlist files, content encryption keys, and video segment files.
- the video data in the file system may be not stored in a real non-volatile storage device; instead the data may be generated only when necessary due to a read operation in request for particular video content from a client device.
- the video source or sources 40 are external to the system and coupled to the streaming video server 50 to provide coded audio-video streams via a secure channel, such as a secure socket layer (SSL) connection, a private network, a virtual private network or other secure connection.
- the video source or sources 40 can have two types of content: live and pre-recorded.
- the video source 40 can be a live video source such as a broadcast cable network, a broadcast satellite network, a broadcast television network, an IP television distribution system, a broadcast mobile network, a video conferencing service or other source of live video.
- Example of video sources 40 of prerecorded content include a video on demand source such as a YouTube, Hulu, Netflix, or other streaming video source, a cable or IP television video on demand network, a digital video recorder, UPnP media server, camcorder, personal computer or other source of stored video.
- the two types of content can be handled differently when limited network bandwidth causes the rate of data transfer to the client to be lower than the encoding rate.
- the streaming system discards portions of the audio-video stream that could not be transferred within some time limit. This ensures that the delay between transmission at the source, and decoding and playback at the client is bounded.
- the streaming video server 50 transcodes or decodes and re-encodes the content from a video source 40 to the bitrates corresponding to the playlists. In this fashion, encoding of a video program is performed only when it is needed in response to a client's request for that particular program. Therefore, videos that are made available by the streaming video server 50 , but never requested by the client, do not incur the overhead of encoding.
- a video encoder of streaming video server 50 When streaming is initiated, a video encoder of streaming video server 50 is allocated for the session. The encoder outputs a continuous audio-video bitstream and then a post-processing step breaks the streaming into the required segments.
- the encryption key generation process can be performed on-demand. This allows the keys to not be saved to a storage device, protecting the security of the key from being read without authorization. For increased content security, the keys can be generated randomly and are never used for more than one streaming session. The encryption process can also be performed on-demand.
- Streaming video server 50 can further include one or more features of the present invention described further in conjunction with FIGS. 2-9 that follow.
- FIG. 2 presents a block diagram representation of streaming server 50 in accordance with an embodiment of the present invention.
- streaming video server 50 includes a command handler 100 , one or more video encoders 104 and video encrypters 106 , and an encryption key generator 108 .
- Streaming video server 50 also includes virtual file system 110 that stores playlist data 112 that includes a content index corresponding to the video programs available from the video source or sources 40 as well as a segment playlist for each video program that includes a list of addresses for each segment of that video program.
- Virtual file system 110 further stores variant playlist data 114 that indicates a plurality of bit rates for streaming each of the plurality of video programs.
- the URIs in the content index, variant playlist, and segment playlist do not correspond to actual files as in a traditional web server or streaming server that serves files stored on hard drive. These addresses are generated by the streaming video server 50 and recognized as references to the virtual files when the client makes the HTTP request for a particular video program. This applies to the URIs for the lists themselves and also for the encrypted segment data and encryption keys.
- the server When a request is received for a “file” in the virtual file system, the server generates the appropriate response depending on the actual request.
- a network interface 102 is included to bidirectionally communicate with at least one client device via network 30 and to bidirectionally communicate with the at least one video source, such as via a secure channel of network 30 or another network.
- the network interface 102 can include a modem, transceiver or other network interface adaptor that implements a serial or parallel connection such as an Ethernet connection, Universal Serial Bus (USB) connection, an Institute of Electrical and Electronics Engineers (IEEE) 1394 (Firewire) connection, small computer serial interface (SCSI), high definition media interface (HDMI) connection or other wired connection that operates in accordance with either a standard or custom interface protocol.
- the network interface 102 can include a wireless link for coupling to the network 30 and/or video source or sources 40 either directly or indirectly through one or more devices that operate in accordance with a wireless network protocol such as 802.11a,b,g,n (referred to generically as 802.11x), Bluetooth, Ultra Wideband (UWB), 3G wireless data connection, 4G wireless data connection or other wireless connection that operates in accordance with either a standard or custom interface protocol.
- a wireless network protocol such as 802.11a,b,g,n (referred to generically as 802.11x), Bluetooth, Ultra Wideband (UWB), 3G wireless data connection, 4G wireless data connection or other wireless connection that operates in accordance with either a standard or custom interface protocol.
- the command handler 100 coordinates the encoding and encryption processes with a client's requests for video content. For example, the command handler 100 receives a HTTP request for a selected one of the plurality of video programs from the at least one client device via the network interface 102 . In response to the request, the selected one of the plurality of video programs is retrieved from the video source or sources 40 , via the network interface. For example, the command handler can access the playlist data 112 to determine the universal resource identifier (URI) or other address for retrieving the selected one of the plurality of video programs from the video source or sources 40 .
- URI universal resource identifier
- the command handler 100 retrieves the selected one of the plurality of video programs.
- a plurality of encoded segments are generated from selected video program, via a video encoder 104 , at a selected bit rate.
- the video encoder 104 can include one or more encoders or transcoders that receive an encoded video program from the video source or sources 40 and that decodes and re-encodes or otherwise transcodes the video program into the scale, resolution, and digital compression format, and at the selected bitrate for the requesting client device.
- the video encoder 104 can operate in accordance with a motion picture expert group (MPEG) format such as MPEG2, MPEG4 part 10, also referred to as H.264.
- MPEG motion picture expert group
- the video source or sources 40 contain unsegmented videos.
- the streaming video server 50 starts the retrieval of the video from the corresponding video source 40 , and performs the encoding and segmentation.
- the segmentation can be done before or after the encoding, though it can be more practical to do the encoding first, then the segmentation.
- Encrypted segments are generated from the encoded segments, via the video encrypter 106 , based on an encryption key.
- the encrypted segments are then stored as video segments 116 that are then sent, via the network interface 102 , as a streaming video signal to the at least one client device that requested the video program.
- the video encrypter 106 can operate in accordance with one or more cryptography algorithms such as data encryption standard (DES), Rivest, Shamir, Adelman (RSA), elliptical curve cryptography (ECC), Advanced Encryption Standard (AES) or other algorithm.
- Streaming server module 50 includes an encryption key generator that generates the encryption key in response to the request.
- the encryption key generator 108 generates encryption keys on-demand. This allows the keys to not be saved to a storage device, protecting the security of the key from being read without authorization. For increased content security, the keys can be generated randomly and are used for only one streaming session. In theory, while identical encryption keys could reoccur randomly, each encryption key is nevertheless practically or substantially unique to the request.
- the command handler 100 generates the selected bit rate, as one of the plurality of bit rates of the variant playlist corresponding to the selected one of the plurality of video programs.
- the command handler 100 obtains channel information from network 30 pertaining to a communication channel between the network interface and the client device that generated the request.
- the channel information can include a channel type, such as 3G, 4G, cable modem, 10base100 Ethernet, digital subscriber line or other network type that has an associated maximum bit rate or other expected bit rate.
- the channel information includes data that indicates current channel conditions, such as current available or actual bit rates, or other indicators of current conditions that can be used to generate a current bit rate.
- the command handler 100 further adjusts the selected bit rate, based on channel information pertaining to a communication channel between the network interface and the requesting client device to adapt to current channel conditions on, for example, a segment by segment basis. As the bit rate is adjusted, the video encoder 106 encodes each current segment at the selected rate.
- the command handler 100 , video encoder 104 , video encrypter 106 , encryption key generator 108 and virtual file system can be implemented via one or more processing devices and one or more memory devices.
- processing devices can include a microprocessor, micro-controller, digital signal processor, microcomputer, central processing unit, field programmable gate array, programmable logic device, state machine, logic circuitry, analog circuitry, digital circuitry, and/or any device that manipulates signals (analog and/or digital) based on operational instructions that are stored in a memory.
- the memory can include a hard disc drive or other disc drive, read-only memory, random access memory, volatile memory, non-volatile memory, static memory, dynamic memory, flash memory, cache memory, and/or any device that stores digital information.
- the memory storing the corresponding operational instructions may be embedded within, or external to, the circuitry comprising the state machine, analog circuitry, digital circuitry, and/or logic circuitry.
- the streaming video server has additional advantages over convention servers. No persistent storage is required for the video segments, variant playlist, and playlists. This is especially important for embedded systems with limited storage capabilities.
- the encryption process is performed on-demand like the encoding process, the key can be protected such that it can be accessed only by the intended client device, as opposed to all potential clients.
- the key itself can be changed while streaming is in progress, and this decision can be made during run-time as needed.
- These content protect features are especially important for premium content such as pay-per-view video.
- the risk of unauthorized access to unencrypted video streams is reduced since they are never stored in encrypted or unencrypted form to a persistent storage device where they may be accessed by attackers.
- live and pre-recorded video content can be handled differently. Streams from live sources will not suffer large delays.
- FIG. 3 presents a block diagram representation of playlist data 112 in accordance with an embodiment of the present invention.
- FIG. 4 presents a block diagram representation of variant playlist data 114 in accordance with an embodiment of the present invention.
- playlist data 112 includes content index 120 a title of all available video selections from any of video sources 40 , and a URI address for accessing the variant playlist data 114 for each program.
- the content index does not have to be in a particular format, as long as the client devices and the streaming video server 50 use the same format or the formats can be translated to be compatible.
- Example formats include JSON, XML, plain text, etc. Other address formats could likewise employed including a file addressing scheme, an IP address, or other logical or physical addressing.
- live video programming could be retrieved, segmented, encoded and encrypted on demand for delivery in response to a client request.
- playlist data 112 includes a segment playlist 122 for each video program and for each bitrate.
- Each segment playlist includes a segment number and address for each segment of the video program and optionally an address of an encryption key used for encrypting that particular segment.
- URIs are shown, other address formats could likewise be employed including a file addressing scheme, an IP address, or other logical or physical addressing.
- the virtual file system also contains variant playlists for each video program in the content index.
- Each variant playlist includes an address (such as the URI shown) for the segment playlists, which are also contained in the virtual file system, covering a wide range of possible bitrates.
- the bitrates cover the cases of streaming over a local-area network (high bitrates), internet and mobile networks (lower bitrates).
- the streaming video server maintains playlist data 112 that includes a content Index that is served to the client upon its request.
- the streaming video server 50 can, for example, query each of the video sources 40 or its list of videos, then aggregate them to the create such a content index.
- the content index contains, for each item, the URI of the variant playlist.
- the client selects one item (e.g., via user input), then makes the HTTP request for the URI corresponding to that variant playlist for that item.
- the variant playlist contains the segment playlist URI for each bitrate, as shown in FIG. 4 .
- the client makes an HTTP request for the selected bitrate's playlist (first on list by default).
- the segment playlist contains the URIs for each segment's data and each segment's encryption key.
- the URIs in the content index, variant playlist, and segment playlist do not correspond to actual files as in a traditional web server or streaming server that serves files stored on hard drive; the URIs are generated by the server and recognized as references to the virtual files when the client makes the HTTP request. This applies to the URIs for the lists themselves and also for the segment data and encryption keys.
- the server When a request is received for a “file” in the virtual file system, the server generates the appropriate response depending on the actual request.
- the variant playlist data includes a plurality of selectable bit rates for a particular title, a group of titles, for all programs from a particular video source 40 or from all video sources. While 6 different bitrates are shown, a greater or fewer number can be implemented including additional or alternative bitrates not expressly shown.
- the streaming video server 50 provides additional advantages over a conventional HLS-based server system.
- Multiple video encoders 104 can be present in order to meet the simultaneous use requirements of on-demand transcoding for multiple client devices.
- the encoded data is generated on as-needed basis and never stored persistently, saving storage.
- Encryption keys are generated on an as-needed basis and never stored except temporarily in volatile memory, also saving storage. Encoded data and encryption keys are never stored permanently, improving content security. Also, live and pre-recorded content can be handled differently, resulting in improved user experience.
- a conventional HLS-based server system must store the video for all bitrate settings.
- the requirement for storage capacity grows with the number of bitrate settings and duration of the video.
- the data must be stored even though it may never be requested by the client, because it can't be absolutely determined at encoding time which particular segments will be used by the clients.
- the data resulting from video transfer from a video source 40 , encoding, key generation, and encryption are not stored to persistent storage (e.g. a hard drive). No transfer, encoding, segmentation, and encryption is done on a source video until that video is requested by the client.
- bitrates for the video must be set before the encoding process starts.
- Some knowledge of the expected network capacity and playback device types is required in order to select the appropriate bitrates.
- three bitrates are used: one low bitrate setting of approximately 64 kbps, and two higher settings between 100 and 500 kbps. Setting the bitrates at encoding time means that the target use cases are also set. Any unexpected cases (e.g., very high network capacity, a new client device) are not addressed.
- an encryption key must be selected that is to be used by all clients.
- the streaming video server 50 can employ a wider range of bitrates without having to encode and store the encoded and encrypted video segments at all of those bitrates prior to streaming.
- prior to streaming little needs to be known about the client device and the possible network conditions. Support can be added for all bitrates that can be reasonable expected, and those that are not used incur no cost in terms of encoding, which is typically a time-consuming and processing-intensive task.
- the bitrate settings might be selected to include the high rates expected when streaming over a local network, and also the relatively low rates expected when streaming over the internet.
- the bitrate values can be selected with a fine granularity over the full range required.
- bitrates allow for the network usage to be optimized for the current conditions. That is, it is more likely that the selected bitrate is the maximum allowed by the network connection between the server and client, thus maximizing the video and audio quality for the end user.
- FIG. 5 presents a block diagram representation of an encoder 104 in accordance with an embodiment of the present invention.
- the video encoder 104 encodes the selected video program 200 into encoded video program segments 204 .
- video encoder 104 encodes based on one or more encoding parameters that are adjusted in response to context data 202 that relates to the streaming session for the selected video program.
- video encoder 104 can operate in accordance with a motion picture expert group (MPEG) format such as MPEG2, MPEG4 part 10, also referred to as H.264.
- MPEG motion picture expert group
- Other video formats such as VC1, digital video (DV), etc., could likewise be employed.
- the video program 200 can be segmented before or after encoding.
- Video encoder 104 can be implemented via one or more processing devices and one or more memory devices.
- Such processing devices can include a microprocessor, micro-controller, digital signal processor, microcomputer, central processing unit, field programmable gate array, programmable logic device, state machine, logic circuitry, analog circuitry, digital circuitry, and/or any device that manipulates signals (analog and/or digital) based on operational instructions that are stored in a memory.
- the memory can include a hard disc drive or other disc drive, read-only memory, random access memory, volatile memory, non-volatile memory, static memory, dynamic memory, flash memory, cache memory, and/or any device that stores digital information.
- the memory storing the corresponding operational instructions may be embedded within, or external to, the circuitry comprising the state machine, analog circuitry, digital circuitry, and/or logic circuitry.
- the streaming server 50 responds to the segment requests by encoding the source at the selected bitrate.
- the encoding process is performed on-demand, therefore the encoding parameters can be adjusted to conform with the context data 202 that reflects the context of the current streaming session.
- the video encoder 104 can adaptively choose or otherwise adjust the overall average bitrate of a segment, and for a selected overall average bitrate, the average bitrate of the audio content and the average bitrate of the video content.
- the video encoder 104 can adaptively choose or otherwise adjust other encoding parameters such as the output video resolution, the output video frame rate, the number of encoded audio channels, etc.
- the context data 202 can indicate information derived from the video program 200 or the video source 40 such as the genre of video (e.g., news, sit com, music, cartoon, video conference), the number of audio channels and/or other information regarding the characteristics, properties or format of the video program 200 . Further, the network interface 102 can receive context data 202 from the client device 60 , 62 , 64 , 66 or 68 that indicates information regarding that client device.
- the genre of video e.g., news, sit com, music, cartoon, video conference
- the network interface 102 can receive context data 202 from the client device 60 , 62 , 64 , 66 or 68 that indicates information regarding that client device.
- Examples of such context data 102 include a current one of a plurality of device states, such as a client device operating system, video display window size (minimized, partial or full screen) or a volume setting that can be used to adjust the video resolution, or audio encoding depth, the number of audio channels or be used to decide whether to or not to include audio information in the stream.
- a client device operating system such as a client device operating system, video display window size (minimized, partial or full screen) or a volume setting that can be used to adjust the video resolution, or audio encoding depth, the number of audio channels or be used to decide whether to or not to include audio information in the stream.
- the context data 202 can also include bandwidth assessment data generated in response to a bandwidth test performed in response to the request for the selected video program 200 .
- a channel bandwidth test can be performed prior to transmission of the first segment.
- the client device 60 , 62 , 64 , 66 or 68 can request a URI that is specifically used to perform the channel bandwidth test.
- the streaming server 50 recognizes this request and responds with a test signal to the requesting client device.
- the transfer rate of the test signal can be measured by the streaming server 50 .
- the response to the variant playlist request can be padded with content that is to be ignored by the requesting client device (normal variant playlist content still exists). This can be done by placing whitespace characters at the end, or inserting lines that start with the comment character ‘#’.
- the amount of data can be chosen so the total size of the variant playlist is suitable for a channel bandwidth test while reducing the time delay in starting the playback of the selected video program to the user. Using too much data could slow the user experience, while using too little data could reduce the accuracy of the test result.
- the transfer rate of the variant playlist is measured by the streaming server 50 .
- FIG. 6 presents a block diagram representation of an encoder 104 in accordance with another embodiment of the present invention.
- an entropy pre-processing module 210 is included that generates entropy preview data 212 from the selected video program 200 .
- the entropy preview data 212 is included in the context data 212 optionally along with other context data 214 , such as any of the previously discussed context data 212 .
- the entropy preview data 212 includes audio entropy data and/or video entropy data of the frames, audio or other portions of the video program 200 about to be encoded by video encoder 104 .
- the streaming server 50 may choose to lower that bitrate based on the entropy preview data 212 .
- Video preprocessing module 210 can be implemented via one or more processing devices and one or more memory devices.
- processing devices can include a microprocessor, micro-controller, digital signal processor, microcomputer, central processing unit, field programmable gate array, programmable logic device, state machine, logic circuitry, analog circuitry, digital circuitry, and/or any device that manipulates signals (analog and/or digital) based on operational instructions that are stored in a memory.
- the memory can include a hard disc drive or other disc drive, read-only memory, random access memory, volatile memory, non-volatile memory, static memory, dynamic memory, flash memory, cache memory, and/or any device that stores digital information.
- entropy preprocessing module 210 can be implemented as part of video encoder 104 .
- the operation of the video encoder 104 can be described in conjunction with the following example where the selected video program 200 is a sports program, characterized by a very low entropy video signal from the 20 second mark to the 25 second mark and five audio channels in the source media.
- the client device is a mobile device (e.g., smartphone) with its video window is maximized, but the client audio is muted after 30 seconds.
- a bandwidth test indicates a capacity of 250 kbits/second.
- Context data 212 reflecting this information is presented to video encoder 104 that responds by adjusting the encoding parameters as follows.
- the initial streaming bitrate is set to 225 kbits/second (10% margin from bandwidth test result) with 25 kbits/second allocated for the audio signal and 200 kbits/second allocated for the video signal (video has higher allocation than audio due to fast motion in typical sports programs).
- An output video resolution of 640 pixels by 480 pixels is selected (because possible higher resolution of the source stream may not be supported by the mobile client, or consume unnecessary processing power to decode and scale down to the native screen resolution).
- the output audio is selected to be two channels (because the mobile client is unlikely to benefit from 5-channel audio).
- the video frame rate is adjusted to 30 frames per second since lower frame rates significantly reduce the quality of typical sports program due to their fast motion.
- the video encoder 104 For the segment containing the low entropy portion of the video ( 20 s - 25 s region), the video encoder 104 lowers the actual bitrate to 125 kbits/second although the client has requested a higher rate. At the 30 second mark when the client audio is muted, the video encoder 104 allocates the total bitrate to the video signal and the audio signal is removed. The previous allocation is restored when feedback from the client device 60 , 62 , 64 , 66 or 68 received via the network interface 102 indicates that the client has set the audio control to a non-zero value.
- One advantage of this approach is a more intelligent use of network bandwidth.
- the actual bitrate of the transmitted signal is lower than the bitrate requested by the client. This minimizes the probably that network congestion will have a negative impact on the playback quality. It also minimizes the user's cost of network usage by lowering the amount of data transmitted when the streamed content does not necessitate the requested bitrate due to low entropy. This approach also improves the user experience. The perceived quality of the received stream is maximized based on the streaming session's context.
- these techniques avoid “ramp up” in visual quality of the received stream that can occur due to the static selection of the initial bitrate.
- the initial bitrate can be chosen conservatively. This can lead to a scenario where the client requests new segments with successively higher bitrates until the channel capacity is close to maximized. The result is a video that starts with low quality and then increases in quality at the segment boundaries in a noticeable and potentially distracting way.
- the ramp up effect can be avoided.
- FIG. 7 presents a flowchart representation of a method in accordance with an embodiment of the present invention.
- a method is shown for use in conjunction with one or more functions and features described in conjunction with FIGS. 1-6 .
- playlist data is stored corresponding to a plurality of video programs available from at least one video source and variant playlist data corresponding to each of the plurality of video programs that indicates a plurality of bit rates for streaming each of the plurality of video programs.
- a request for a selected one of the plurality of video programs is received from at least one client device.
- the selected one of the plurality of video program is retrieved from the at least one video source in response to the request.
- step 406 the selected one of the plurality of video program is segmented into a plurality of segments.
- step 408 a plurality of encoded segments is generated from the plurality of segments, based on at least one encoding parameter that is adjusted in response to context data relating to the streaming session for the selected one of the plurality of video programs.
- the context data can indicate a genre of the selected one of the plurality of video programs and/or a number of audio channels included in the selected one of the plurality of video programs.
- the context data can include bandwidth assessment data generated in response to a bandwidth test performed in response to the request for the selected one of the plurality of video programs.
- the adjusted encoding parameters can include an overall average bitrate for each of the plurality of encoded segments; an average video bitrate for each of the plurality of encoded segments; an average audio bitrate for each of the plurality of encoded segments; a video resolution for each of the plurality of encoded segments; a video frame rate for each of the plurality of encoded segments; and/or a number of encoded audio channels for each of the plurality of encoded segments.
- FIG. 8 presents a flowchart representation of a method in accordance with an embodiment of the present invention.
- a method is shown for use in conjunction with one or more functions and features described in conjunction with FIGS. 1-7 .
- context data is received from the client device.
- the context data can indicates a current one of a plurality of device states, such as a video display window size, and a volume setting.
- the context data can indicate an operating system of the client device.
- FIG. 9 presents a flowchart representation of a method in accordance with an embodiment of the present invention.
- a method is shown for use in conjunction with one or more functions and features described in conjunction with FIGS. 1-8 .
- entropy preview data is generated by preprocessing the selected one of the plurality of video programs.
- the entropy preview data can be included in the context data.
- the entropy preview data can include audio entropy data and/or video entropy data.
- FIG. 10 presents a flowchart representation of a method in accordance with an embodiment of the present invention. In particular, a method is shown for use in conjunction with one or more functions and features described in conjunction with FIGS. 1-9 .
- step 430 encrypted segments are generated from the plurality of encoded segments in accordance with an encryption key.
- step 432 a streaming video signal is sent to the at least one client device that includes the plurality of encrypted segments.
- optional circuit components can be implemented using 0.35 micron or smaller CMOS technology. Provided however that other circuit technologies, both integrated or non-integrated, may be used within the broad scope of the present invention.
- the term “substantially” or “approximately”, as may be used herein, provides an industry-accepted tolerance to its corresponding term and/or relativity between items. Such an industry-accepted tolerance ranges from less than one percent to twenty percent and corresponds to, but is not limited to, component values, integrated circuit process variations, temperature variations, rise and fall times, and/or thermal noise. Such relativity between items ranges from a difference of a few percent to magnitude differences.
- the term “coupled”, as may be used herein, includes direct coupling and indirect coupling via another component, element, circuit, or module where, for indirect coupling, the intervening component, element, circuit, or module does not modify the information of a signal but may adjust its current level, voltage level, and/or power level.
- inferred coupling i.e., where one element is coupled to another element by inference
- inferred coupling includes direct and indirect coupling between two elements in the same manner as “coupled”.
- the term “compares favorably”, as may be used herein, indicates that a comparison between two or more elements, items, signals, etc., provides a desired relationship. For example, when the desired relationship is that signal 1 has a greater magnitude than signal 2 , a favorable comparison may be achieved when the magnitude of signal 1 is greater than that of signal 2 or when the magnitude of signal 2 is less than that of signal 1 .
- a module includes a functional block that is implemented in hardware, software, and/or firmware that performs one or module functions such as the processing of an input signal to produce an output signal.
- a module may contain submodules that themselves are modules.
Abstract
Description
- The present U.S. Utility Patent Application claims priority pursuant to 35 U.S.C. §120 as a continuation of U.S. Utility application Ser. No. 13/197,400, entitled “ADAPTIVE VIDEO SERVER WITH VIRTUAL FILE SYSTEM AND METHODS FOR USE THEREWITH”, filed Aug. 3, 2011, which is a continuation-in-part of U.S. Utility application Ser. No. 13/095,060, entitled “STREAMING VIDEO SERVER WITH VIRTUAL FILE SYSTEM AND METHODS FOR USE THEREWITH”, filed Apr. 27, 2011, issued as U.S. Pat. No. 8,510,555 on Aug. 1, /2013, all of which are hereby incorporated herein by reference in their entirety and made part of the present U.S. Utility Patent Application for all purposes.
- The present invention relates to servers used for streaming media applications including video on demand.
- The number of households having multiple television sets is increasing, and many users want the latest and greatest video viewing services. As such, many households have satellite receivers, cable set-top boxes, and televisions, et cetera, that provide access to broadcast video services. For in-home Internet access, each computer or Internet device can have its own Internet connection. As such, each computer or Internet device includes a modem. As an alternative, an in-home wireless local area network may be used to provide Internet access and to communicate multimedia information to multiple devices within the home. In such an in-home local area network, each computer or Internet device includes a network card to access an IP gateway. The gateway provides the coupling to the Internet. As an alternative to broadcast video, the Internet provides access to streaming video services. Instead of broadcasting all programming to every customer, each customer receives only those programs that they want, when they want them.
- One of the most common ways of streaming video over the Internet today is through the use of the “HTTP Live Streaming” (HLS) protocol. It is developed by Apple Corporation and supported in most of Apple's devices. The HLS protocol operates on a streaming server that uses a standard HTTP (Hypertext Transfer Protocol) web server component. A video encoder takes a source video and encodes it at different bitrates intended for use in different network conditions (high or low capacity) and with different playback devices. A variant playlist is stored on the server's file system that describes one video that is available in multiple bitrates. For each available bitrate, a server-side tool takes the encoder's output file and breaks it into segments of equal duration (usually 10 seconds); these segments are stored on the server's file system. The segments may be optionally encrypted.
- For each available bitrate, a playlist is stored on the server's file system. The playlist specifies the properties (most importantly duration and URI) of the available segments. The encoded video, variant playlists and playlists must be prepared and available to the web server prior to streaming. The client initiates streaming by first requesting the variant playlist over HTTP. The client requests the first playlist listed in the variant playlist. The client requests the first segment in the playlist, then the second, and so on. Depending on the transfer rate of a segment, the client will choose to switch to a playlist for a higher or lower bitrate, and proceed to request the video segments from the new playlist. The received segments are decoded and displayed by the client as one continuous video stream.
- An advantage of HLS streaming is that the server is a standard web server. Beyond the application-specific tools needed for segmentation and playlist generation, the software and infrastructure comprises of standard and freely available components that are often already in place. The limitations and disadvantages of HLS and other conventional and traditional approaches will become apparent to one of ordinary skill in the art through comparison of such systems with the present invention.
-
FIG. 1 presents a pictorial representation of a content distribution system that in accordance with an embodiment of the present invention. -
FIG. 2 presents a block diagram representation ofstreaming server 50 in accordance with an embodiment of the present invention. -
FIG. 3 presents a block diagram representation ofplaylist data 112 in accordance with an embodiment of the present invention. -
FIG. 4 presents a block diagram representation ofvariant playlist data 114 in accordance with an embodiment of the present invention. -
FIG. 5 presents a block diagram representation of anencoder 104 in accordance with an embodiment of the present invention. -
FIG. 6 presents a block diagram representation of anencoder 104 in accordance with another embodiment of the present invention. -
FIG. 7 presents a flowchart representation of a method in accordance with an embodiment of the present invention. -
FIG. 8 presents a flowchart representation of a method in accordance with an embodiment of the present invention. -
FIG. 9 presents a flowchart representation of a method in accordance with an embodiment of the present invention. -
FIG. 10 presents a flowchart representation of a method in accordance with an embodiment of the present invention. -
FIG. 1 presents a pictorial representation of a content distribution system in accordance with an embodiment of the present invention. In particular, astreaming video server 50 is capable of accessing and distributing content from one ormore video sources 40 to a plurality of client devices such as atelevision 60, smart phone, internet tablet or otherpersonal media players handheld video player 66, andpersonal computer 68. While specific client devices are shown,video server 50 is capable of accessing and distributing content from one ormore video sources 40 to other client devices that can receive and reproduce media content. - The
streaming video server 50 uses an on-demand encoding process and a virtual file system. In an embodiment of the present invention, thestreaming video server 50 allows many bitrates to be supported with no additional requirements for persistent storage. Thestreaming video server 50 accesses a virtual file system containing the files needed for streaming: variant playlist files, playlist files, content encryption keys, and video segment files. The video data in the file system may be not stored in a real non-volatile storage device; instead the data may be generated only when necessary due to a read operation in request for particular video content from a client device. - In the example shown, the video source or
sources 40 are external to the system and coupled to thestreaming video server 50 to provide coded audio-video streams via a secure channel, such as a secure socket layer (SSL) connection, a private network, a virtual private network or other secure connection. The video source orsources 40 can have two types of content: live and pre-recorded. Thevideo source 40 can be a live video source such as a broadcast cable network, a broadcast satellite network, a broadcast television network, an IP television distribution system, a broadcast mobile network, a video conferencing service or other source of live video. Example ofvideo sources 40 of prerecorded content include a video on demand source such as a YouTube, Hulu, Netflix, or other streaming video source, a cable or IP television video on demand network, a digital video recorder, UPnP media server, camcorder, personal computer or other source of stored video. The two types of content can be handled differently when limited network bandwidth causes the rate of data transfer to the client to be lower than the encoding rate. For live content, the streaming system discards portions of the audio-video stream that could not be transferred within some time limit. This ensures that the delay between transmission at the source, and decoding and playback at the client is bounded. For a video conference, this means that conversations will not be hampered by excessive delays, but there may be discontinuities in the conversation when the network slows suddenly. Pre-recorded content can be streamed in a continuous manner. If the network slows suddenly and the client runs out of data to decode, it handles this in the short term by waiting for more data, and in the longer term by selecting a variant playlist with a lower bitrate. - In operation, the
streaming video server 50 transcodes or decodes and re-encodes the content from avideo source 40 to the bitrates corresponding to the playlists. In this fashion, encoding of a video program is performed only when it is needed in response to a client's request for that particular program. Therefore, videos that are made available by thestreaming video server 50, but never requested by the client, do not incur the overhead of encoding. - When streaming is initiated, a video encoder of streaming
video server 50 is allocated for the session. The encoder outputs a continuous audio-video bitstream and then a post-processing step breaks the streaming into the required segments. In addition, the encryption key generation process can be performed on-demand. This allows the keys to not be saved to a storage device, protecting the security of the key from being read without authorization. For increased content security, the keys can be generated randomly and are never used for more than one streaming session. The encryption process can also be performed on-demand. - Streaming
video server 50 can further include one or more features of the present invention described further in conjunction withFIGS. 2-9 that follow. -
FIG. 2 presents a block diagram representation of streamingserver 50 in accordance with an embodiment of the present invention. As shown, streamingvideo server 50 includes acommand handler 100, one ormore video encoders 104 andvideo encrypters 106, and anencryption key generator 108. Streamingvideo server 50 also includesvirtual file system 110 that storesplaylist data 112 that includes a content index corresponding to the video programs available from the video source orsources 40 as well as a segment playlist for each video program that includes a list of addresses for each segment of that video program.Virtual file system 110 further storesvariant playlist data 114 that indicates a plurality of bit rates for streaming each of the plurality of video programs. The URIs in the content index, variant playlist, and segment playlist do not correspond to actual files as in a traditional web server or streaming server that serves files stored on hard drive. These addresses are generated by thestreaming video server 50 and recognized as references to the virtual files when the client makes the HTTP request for a particular video program. This applies to the URIs for the lists themselves and also for the encrypted segment data and encryption keys. When a request is received for a “file” in the virtual file system, the server generates the appropriate response depending on the actual request. For example, (a) responding with the content index, variant playlist, segment playlist data using information stored in memory, (b) generating a random encryption key, storing it to memory (for subsequent segment requests), and sending it in the response to the client, (c), initiating video transfer from a source, initiating the encoding and segmentation, sending the first segment in the response to the client, (d) for a previously initiated transfer, encoding, segmenting, and return the next segment to the client. - A
network interface 102 is included to bidirectionally communicate with at least one client device vianetwork 30 and to bidirectionally communicate with the at least one video source, such as via a secure channel ofnetwork 30 or another network. Thenetwork interface 102 can include a modem, transceiver or other network interface adaptor that implements a serial or parallel connection such as an Ethernet connection, Universal Serial Bus (USB) connection, an Institute of Electrical and Electronics Engineers (IEEE) 1394 (Firewire) connection, small computer serial interface (SCSI), high definition media interface (HDMI) connection or other wired connection that operates in accordance with either a standard or custom interface protocol. In addition or in the alternative, thenetwork interface 102 can include a wireless link for coupling to thenetwork 30 and/or video source orsources 40 either directly or indirectly through one or more devices that operate in accordance with a wireless network protocol such as 802.11a,b,g,n (referred to generically as 802.11x), Bluetooth, Ultra Wideband (UWB), 3G wireless data connection, 4G wireless data connection or other wireless connection that operates in accordance with either a standard or custom interface protocol. - In operation, the
command handler 100 coordinates the encoding and encryption processes with a client's requests for video content. For example, thecommand handler 100 receives a HTTP request for a selected one of the plurality of video programs from the at least one client device via thenetwork interface 102. In response to the request, the selected one of the plurality of video programs is retrieved from the video source orsources 40, via the network interface. For example, the command handler can access theplaylist data 112 to determine the universal resource identifier (URI) or other address for retrieving the selected one of the plurality of video programs from the video source or sources 40. - The
command handler 100 retrieves the selected one of the plurality of video programs. A plurality of encoded segments are generated from selected video program, via avideo encoder 104, at a selected bit rate. Thevideo encoder 104 can include one or more encoders or transcoders that receive an encoded video program from the video source orsources 40 and that decodes and re-encodes or otherwise transcodes the video program into the scale, resolution, and digital compression format, and at the selected bitrate for the requesting client device. In particular, thevideo encoder 104 can operate in accordance with a motion picture expert group (MPEG) format such as MPEG2, MPEG4 part 10, also referred to as H.264. Other video formats such as VC1, digital video (DV), etc., could likewise be employed. In an embodiment of the present invention, the video source orsources 40 contain unsegmented videos. Upon the client request of the first segment for a particular video and bitrate, thestreaming video server 50 starts the retrieval of the video from the correspondingvideo source 40, and performs the encoding and segmentation. The segmentation can be done before or after the encoding, though it can be more practical to do the encoding first, then the segmentation. - Encrypted segments are generated from the encoded segments, via the
video encrypter 106, based on an encryption key. The encrypted segments are then stored asvideo segments 116 that are then sent, via thenetwork interface 102, as a streaming video signal to the at least one client device that requested the video program. Thevideo encrypter 106 can operate in accordance with one or more cryptography algorithms such as data encryption standard (DES), Rivest, Shamir, Adelman (RSA), elliptical curve cryptography (ECC), Advanced Encryption Standard (AES) or other algorithm. -
Streaming server module 50 includes an encryption key generator that generates the encryption key in response to the request. As discussed in conjunction withFIG. 1 , theencryption key generator 108 generates encryption keys on-demand. This allows the keys to not be saved to a storage device, protecting the security of the key from being read without authorization. For increased content security, the keys can be generated randomly and are used for only one streaming session. In theory, while identical encryption keys could reoccur randomly, each encryption key is nevertheless practically or substantially unique to the request. - In an embodiment of the present invention, the
command handler 100 generates the selected bit rate, as one of the plurality of bit rates of the variant playlist corresponding to the selected one of the plurality of video programs. For example, thecommand handler 100 obtains channel information fromnetwork 30 pertaining to a communication channel between the network interface and the client device that generated the request. The channel information can include a channel type, such as 3G, 4G, cable modem, 10base100 Ethernet, digital subscriber line or other network type that has an associated maximum bit rate or other expected bit rate. In addition or in the alternative, the channel information includes data that indicates current channel conditions, such as current available or actual bit rates, or other indicators of current conditions that can be used to generate a current bit rate. In addition to setting an initial bit rate, thecommand handler 100 further adjusts the selected bit rate, based on channel information pertaining to a communication channel between the network interface and the requesting client device to adapt to current channel conditions on, for example, a segment by segment basis. As the bit rate is adjusted, thevideo encoder 106 encodes each current segment at the selected rate. - The
command handler 100,video encoder 104,video encrypter 106,encryption key generator 108 and virtual file system can be implemented via one or more processing devices and one or more memory devices. Such processing devices can include a microprocessor, micro-controller, digital signal processor, microcomputer, central processing unit, field programmable gate array, programmable logic device, state machine, logic circuitry, analog circuitry, digital circuitry, and/or any device that manipulates signals (analog and/or digital) based on operational instructions that are stored in a memory. The memory can include a hard disc drive or other disc drive, read-only memory, random access memory, volatile memory, non-volatile memory, static memory, dynamic memory, flash memory, cache memory, and/or any device that stores digital information. Note that when the processing device implements one or more of its functions via a state machine, analog circuitry, digital circuitry, and/or logic circuitry, the memory storing the corresponding operational instructions may be embedded within, or external to, the circuitry comprising the state machine, analog circuitry, digital circuitry, and/or logic circuitry. - The streaming video server has additional advantages over convention servers. No persistent storage is required for the video segments, variant playlist, and playlists. This is especially important for embedded systems with limited storage capabilities. In addition, since the encryption process is performed on-demand like the encoding process, the key can be protected such that it can be accessed only by the intended client device, as opposed to all potential clients. Furthermore, the key itself can be changed while streaming is in progress, and this decision can be made during run-time as needed. These content protect features are especially important for premium content such as pay-per-view video. Further, the risk of unauthorized access to unencrypted video streams is reduced since they are never stored in encrypted or unencrypted form to a persistent storage device where they may be accessed by attackers. Also, live and pre-recorded video content can be handled differently. Streams from live sources will not suffer large delays.
-
FIG. 3 presents a block diagram representation ofplaylist data 112 in accordance with an embodiment of the present invention.FIG. 4 presents a block diagram representation ofvariant playlist data 114 in accordance with an embodiment of the present invention. In the example shown,playlist data 112 includes content index 120 a title of all available video selections from any ofvideo sources 40, and a URI address for accessing thevariant playlist data 114 for each program. The content index does not have to be in a particular format, as long as the client devices and thestreaming video server 50 use the same format or the formats can be translated to be compatible. Example formats include JSON, XML, plain text, etc. Other address formats could likewise employed including a file addressing scheme, an IP address, or other logical or physical addressing. In addition, the while specific video on-demand programs are presented, as previously discussed, live video programming could be retrieved, segmented, encoded and encrypted on demand for delivery in response to a client request. - In addition,
playlist data 112 includes asegment playlist 122 for each video program and for each bitrate. Each segment playlist includes a segment number and address for each segment of the video program and optionally an address of an encryption key used for encrypting that particular segment. Again, while URIs are shown, other address formats could likewise be employed including a file addressing scheme, an IP address, or other logical or physical addressing. - The virtual file system also contains variant playlists for each video program in the content index. Each variant playlist includes an address (such as the URI shown) for the segment playlists, which are also contained in the virtual file system, covering a wide range of possible bitrates. The bitrates cover the cases of streaming over a local-area network (high bitrates), internet and mobile networks (lower bitrates).
- In one example of operation, the streaming video server maintains
playlist data 112 that includes a content Index that is served to the client upon its request. Thestreaming video server 50 can, for example, query each of thevideo sources 40 or its list of videos, then aggregate them to the create such a content index. The content index contains, for each item, the URI of the variant playlist. The client selects one item (e.g., via user input), then makes the HTTP request for the URI corresponding to that variant playlist for that item. The variant playlist contains the segment playlist URI for each bitrate, as shown inFIG. 4 . The client makes an HTTP request for the selected bitrate's playlist (first on list by default). The segment playlist contains the URIs for each segment's data and each segment's encryption key. - The URIs in the content index, variant playlist, and segment playlist do not correspond to actual files as in a traditional web server or streaming server that serves files stored on hard drive; the URIs are generated by the server and recognized as references to the virtual files when the client makes the HTTP request. This applies to the URIs for the lists themselves and also for the segment data and encryption keys. When a request is received for a “file” in the virtual file system, the server generates the appropriate response depending on the actual request. For example, (a) responding with the content index, variant playlist, segment playlist data using information stored in memory, (b) generating a random encryption key, storing it to memory (for subsequent segment requests), and sending it in the response to the client, (c), initiating video transfer from a source, initiating the encoding and segmentation, sending the first segment in the response to the client, (d) for a previously initiated transfer, encoding and segmentation, return the next segment to the client.
- As shown the variant playlist data includes a plurality of selectable bit rates for a particular title, a group of titles, for all programs from a
particular video source 40 or from all video sources. While 6 different bitrates are shown, a greater or fewer number can be implemented including additional or alternative bitrates not expressly shown. - The
streaming video server 50 provides additional advantages over a conventional HLS-based server system.Multiple video encoders 104 can be present in order to meet the simultaneous use requirements of on-demand transcoding for multiple client devices. The encoded data is generated on as-needed basis and never stored persistently, saving storage. Encryption keys are generated on an as-needed basis and never stored except temporarily in volatile memory, also saving storage. Encoded data and encryption keys are never stored permanently, improving content security. Also, live and pre-recorded content can be handled differently, resulting in improved user experience. - A conventional HLS-based server system must store the video for all bitrate settings. The requirement for storage capacity grows with the number of bitrate settings and duration of the video. The data must be stored even though it may never be requested by the client, because it can't be absolutely determined at encoding time which particular segments will be used by the clients. In the present system, the data resulting from video transfer from a
video source 40, encoding, key generation, and encryption are not stored to persistent storage (e.g. a hard drive). No transfer, encoding, segmentation, and encryption is done on a source video until that video is requested by the client. - In addition, in a conventional HLS-based server system, the bitrates for the video must be set before the encoding process starts. Some knowledge of the expected network capacity and playback device types is required in order to select the appropriate bitrates. Typically, three bitrates are used: one low bitrate setting of approximately 64 kbps, and two higher settings between 100 and 500 kbps. Setting the bitrates at encoding time means that the target use cases are also set. Any unexpected cases (e.g., very high network capacity, a new client device) are not addressed. When the video segments are encrypted, an encryption key must be selected that is to be used by all clients.
- In contrast, the
streaming video server 50 can employ a wider range of bitrates without having to encode and store the encoded and encrypted video segments at all of those bitrates prior to streaming. In an embodiment of the present invention, prior to streaming, little needs to be known about the client device and the possible network conditions. Support can be added for all bitrates that can be reasonable expected, and those that are not used incur no cost in terms of encoding, which is typically a time-consuming and processing-intensive task. For example, the bitrate settings might be selected to include the high rates expected when streaming over a local network, and also the relatively low rates expected when streaming over the internet. In addition, since additional bitrate settings can be added for little additional cost, the bitrate values can be selected with a fine granularity over the full range required. This reduces the distracting effect of the video and audio quality changing greatly from one segment to the other due to a large change in bitrate. More importantly, the additional bitrates allow for the network usage to be optimized for the current conditions. That is, it is more likely that the selected bitrate is the maximum allowed by the network connection between the server and client, thus maximizing the video and audio quality for the end user. -
FIG. 5 presents a block diagram representation of anencoder 104 in accordance with an embodiment of the present invention. In this embodiment, thevideo encoder 104 encodes the selectedvideo program 200 into encodedvideo program segments 204. In particular,video encoder 104 encodes based on one or more encoding parameters that are adjusted in response tocontext data 202 that relates to the streaming session for the selected video program. - As discussed in conjunction with
FIG. 2 ,video encoder 104 can operate in accordance with a motion picture expert group (MPEG) format such as MPEG2, MPEG4 part 10, also referred to as H.264. Other video formats such as VC1, digital video (DV), etc., could likewise be employed. In an embodiment of the present invention, thevideo program 200 can be segmented before or after encoding.Video encoder 104 can be implemented via one or more processing devices and one or more memory devices. Such processing devices can include a microprocessor, micro-controller, digital signal processor, microcomputer, central processing unit, field programmable gate array, programmable logic device, state machine, logic circuitry, analog circuitry, digital circuitry, and/or any device that manipulates signals (analog and/or digital) based on operational instructions that are stored in a memory. The memory can include a hard disc drive or other disc drive, read-only memory, random access memory, volatile memory, non-volatile memory, static memory, dynamic memory, flash memory, cache memory, and/or any device that stores digital information. Note that when the processing device implements one or more of its functions via a state machine, analog circuitry, digital circuitry, and/or logic circuitry, the memory storing the corresponding operational instructions may be embedded within, or external to, the circuitry comprising the state machine, analog circuitry, digital circuitry, and/or logic circuitry. - In operation, the streaming
server 50 responds to the segment requests by encoding the source at the selected bitrate. The encoding process is performed on-demand, therefore the encoding parameters can be adjusted to conform with thecontext data 202 that reflects the context of the current streaming session. In response to thecontext data 102, thevideo encoder 104 can adaptively choose or otherwise adjust the overall average bitrate of a segment, and for a selected overall average bitrate, the average bitrate of the audio content and the average bitrate of the video content. In addition, thevideo encoder 104 can adaptively choose or otherwise adjust other encoding parameters such as the output video resolution, the output video frame rate, the number of encoded audio channels, etc. - The
context data 202 can indicate information derived from thevideo program 200 or thevideo source 40 such as the genre of video (e.g., news, sit com, music, cartoon, video conference), the number of audio channels and/or other information regarding the characteristics, properties or format of thevideo program 200. Further, thenetwork interface 102 can receivecontext data 202 from theclient device such context data 102 include a current one of a plurality of device states, such as a client device operating system, video display window size (minimized, partial or full screen) or a volume setting that can be used to adjust the video resolution, or audio encoding depth, the number of audio channels or be used to decide whether to or not to include audio information in the stream. - The
context data 202 can also include bandwidth assessment data generated in response to a bandwidth test performed in response to the request for the selectedvideo program 200. A channel bandwidth test can be performed prior to transmission of the first segment. For example, prior to requesting the first segment, theclient device server 50 recognizes this request and responds with a test signal to the requesting client device. - The transfer rate of the test signal can be measured by the streaming
server 50. For example, the response to the variant playlist request can be padded with content that is to be ignored by the requesting client device (normal variant playlist content still exists). This can be done by placing whitespace characters at the end, or inserting lines that start with the comment character ‘#’. The amount of data can be chosen so the total size of the variant playlist is suitable for a channel bandwidth test while reducing the time delay in starting the playback of the selected video program to the user. Using too much data could slow the user experience, while using too little data could reduce the accuracy of the test result. The transfer rate of the variant playlist is measured by the streamingserver 50. -
FIG. 6 presents a block diagram representation of anencoder 104 in accordance with another embodiment of the present invention. In this embodiment, anentropy pre-processing module 210 is included that generatesentropy preview data 212 from the selectedvideo program 200. Theentropy preview data 212 is included in thecontext data 212 optionally along withother context data 214, such as any of the previously discussedcontext data 212. In operation, theentropy preview data 212 includes audio entropy data and/or video entropy data of the frames, audio or other portions of thevideo program 200 about to be encoded byvideo encoder 104. In particular, while a segment's maximum bitrate may be selected by theclient device server 50 may choose to lower that bitrate based on theentropy preview data 212. -
Video preprocessing module 210 can be implemented via one or more processing devices and one or more memory devices. Such processing devices can include a microprocessor, micro-controller, digital signal processor, microcomputer, central processing unit, field programmable gate array, programmable logic device, state machine, logic circuitry, analog circuitry, digital circuitry, and/or any device that manipulates signals (analog and/or digital) based on operational instructions that are stored in a memory. The memory can include a hard disc drive or other disc drive, read-only memory, random access memory, volatile memory, non-volatile memory, static memory, dynamic memory, flash memory, cache memory, and/or any device that stores digital information. Note that when the processing device implements one or more of its functions via a state machine, analog circuitry, digital circuitry, and/or logic circuitry, the memory storing the corresponding operational instructions may be embedded within, or external to, the circuitry comprising the state machine, analog circuitry, digital circuitry, and/or logic circuitry. While shown as a separate device,entropy preprocessing module 210 can be implemented as part ofvideo encoder 104. - The operation of the
video encoder 104 can be described in conjunction with the following example where the selectedvideo program 200 is a sports program, characterized by a very low entropy video signal from the 20 second mark to the 25 second mark and five audio channels in the source media. Assume further that the client device is a mobile device (e.g., smartphone) with its video window is maximized, but the client audio is muted after 30 seconds. A bandwidth test indicates a capacity of 250 kbits/second.Context data 212 reflecting this information is presented tovideo encoder 104 that responds by adjusting the encoding parameters as follows. The initial streaming bitrate is set to 225 kbits/second (10% margin from bandwidth test result) with 25 kbits/second allocated for the audio signal and 200 kbits/second allocated for the video signal (video has higher allocation than audio due to fast motion in typical sports programs). An output video resolution of 640 pixels by 480 pixels is selected (because possible higher resolution of the source stream may not be supported by the mobile client, or consume unnecessary processing power to decode and scale down to the native screen resolution). The output audio is selected to be two channels (because the mobile client is unlikely to benefit from 5-channel audio). The video frame rate is adjusted to 30 frames per second since lower frame rates significantly reduce the quality of typical sports program due to their fast motion. For the segment containing the low entropy portion of the video (20 s-25 s region), thevideo encoder 104 lowers the actual bitrate to 125 kbits/second although the client has requested a higher rate. At the 30 second mark when the client audio is muted, thevideo encoder 104 allocates the total bitrate to the video signal and the audio signal is removed. The previous allocation is restored when feedback from theclient device network interface 102 indicates that the client has set the audio control to a non-zero value. - One advantage of this approach is a more intelligent use of network bandwidth. When the audio and video signals have low entropy, the actual bitrate of the transmitted signal is lower than the bitrate requested by the client. This minimizes the probably that network congestion will have a negative impact on the playback quality. It also minimizes the user's cost of network usage by lowering the amount of data transmitted when the streamed content does not necessitate the requested bitrate due to low entropy. This approach also improves the user experience. The perceived quality of the received stream is maximized based on the streaming session's context.
- In addition, these techniques avoid “ramp up” in visual quality of the received stream that can occur due to the static selection of the initial bitrate. In the prior art, the initial bitrate can be chosen conservatively. This can lead to a scenario where the client requests new segments with successively higher bitrates until the channel capacity is close to maximized. The result is a video that starts with low quality and then increases in quality at the segment boundaries in a noticeable and potentially distracting way. By selecting the initial bitrate based on the actual channel capacity just prior to the start of the streaming session, the ramp up effect can be avoided.
-
FIG. 7 presents a flowchart representation of a method in accordance with an embodiment of the present invention. In particular, a method is shown for use in conjunction with one or more functions and features described in conjunction withFIGS. 1-6 . Instep 400, playlist data is stored corresponding to a plurality of video programs available from at least one video source and variant playlist data corresponding to each of the plurality of video programs that indicates a plurality of bit rates for streaming each of the plurality of video programs. Instep 402, a request for a selected one of the plurality of video programs is received from at least one client device. Instep 404, the selected one of the plurality of video program is retrieved from the at least one video source in response to the request. Instep 406, the selected one of the plurality of video program is segmented into a plurality of segments. Instep 408, a plurality of encoded segments is generated from the plurality of segments, based on at least one encoding parameter that is adjusted in response to context data relating to the streaming session for the selected one of the plurality of video programs. - As discussed, the ordering of the
steps -
FIG. 8 presents a flowchart representation of a method in accordance with an embodiment of the present invention. In particular, a method is shown for use in conjunction with one or more functions and features described in conjunction withFIGS. 1-7 . Instep 410, context data is received from the client device. The context data can indicates a current one of a plurality of device states, such as a video display window size, and a volume setting. The context data can indicate an operating system of the client device. -
FIG. 9 presents a flowchart representation of a method in accordance with an embodiment of the present invention. In particular, a method is shown for use in conjunction with one or more functions and features described in conjunction withFIGS. 1-8 . Instep 420, entropy preview data is generated by preprocessing the selected one of the plurality of video programs. The entropy preview data can be included in the context data. The entropy preview data can include audio entropy data and/or video entropy data. -
FIG. 10 presents a flowchart representation of a method in accordance with an embodiment of the present invention. In particular, a method is shown for use in conjunction with one or more functions and features described in conjunction withFIGS. 1-9 . Instep 430 encrypted segments are generated from the plurality of encoded segments in accordance with an encryption key. Instep 432, a streaming video signal is sent to the at least one client device that includes the plurality of encrypted segments. - In preferred embodiments, optional circuit components can be implemented using 0.35 micron or smaller CMOS technology. Provided however that other circuit technologies, both integrated or non-integrated, may be used within the broad scope of the present invention.
- As one of ordinary skill in the art will appreciate, the term “substantially” or “approximately”, as may be used herein, provides an industry-accepted tolerance to its corresponding term and/or relativity between items. Such an industry-accepted tolerance ranges from less than one percent to twenty percent and corresponds to, but is not limited to, component values, integrated circuit process variations, temperature variations, rise and fall times, and/or thermal noise. Such relativity between items ranges from a difference of a few percent to magnitude differences. As one of ordinary skill in the art will further appreciate, the term “coupled”, as may be used herein, includes direct coupling and indirect coupling via another component, element, circuit, or module where, for indirect coupling, the intervening component, element, circuit, or module does not modify the information of a signal but may adjust its current level, voltage level, and/or power level. As one of ordinary skill in the art will also appreciate, inferred coupling (i.e., where one element is coupled to another element by inference) includes direct and indirect coupling between two elements in the same manner as “coupled”. As one of ordinary skill in the art will further appreciate, the term “compares favorably”, as may be used herein, indicates that a comparison between two or more elements, items, signals, etc., provides a desired relationship. For example, when the desired relationship is that
signal 1 has a greater magnitude thansignal 2, a favorable comparison may be achieved when the magnitude ofsignal 1 is greater than that ofsignal 2 or when the magnitude ofsignal 2 is less than that ofsignal 1. - As the term module is used in the description of the various embodiments of the present invention, a module includes a functional block that is implemented in hardware, software, and/or firmware that performs one or module functions such as the processing of an input signal to produce an output signal. As used herein, a module may contain submodules that themselves are modules.
- Thus, there has been described herein an apparatus and method, as well as several embodiments including a preferred embodiment, for implementing a media distribution system. While described primarily in terms of video programming, it is understood that the video programming can include associated audio and that the present invention could likewise to be applied to associated audio or the distribution of audio programming that is unassociated with video. Various embodiments of the present invention herein-described have features that distinguish the present invention from the prior art.
- It will be apparent to those skilled in the art that the disclosed invention may be modified in numerous ways and may assume many embodiments other than the preferred forms specifically set out and described above. Accordingly, it is intended by the appended claims to cover all modifications of the invention which fall within the true spirit and scope of the invention.
Claims (20)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14/284,195 US20140331262A1 (en) | 2011-04-27 | 2014-05-21 | Adaptive video server with virtual file system and methods for use therewith |
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/095,060 US8510555B2 (en) | 2011-04-27 | 2011-04-27 | Streaming video server with virtual file system and methods for use therewith |
US13/197,400 US8813116B2 (en) | 2011-04-27 | 2011-08-03 | Adaptive video server with virtual file system and methods for use therewith |
US14/284,195 US20140331262A1 (en) | 2011-04-27 | 2014-05-21 | Adaptive video server with virtual file system and methods for use therewith |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/197,400 Continuation US8813116B2 (en) | 2011-04-27 | 2011-08-03 | Adaptive video server with virtual file system and methods for use therewith |
Publications (1)
Publication Number | Publication Date |
---|---|
US20140331262A1 true US20140331262A1 (en) | 2014-11-06 |
Family
ID=47069009
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/197,400 Active 2031-05-30 US8813116B2 (en) | 2011-04-27 | 2011-08-03 | Adaptive video server with virtual file system and methods for use therewith |
US14/284,195 Abandoned US20140331262A1 (en) | 2011-04-27 | 2014-05-21 | Adaptive video server with virtual file system and methods for use therewith |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/197,400 Active 2031-05-30 US8813116B2 (en) | 2011-04-27 | 2011-08-03 | Adaptive video server with virtual file system and methods for use therewith |
Country Status (4)
Country | Link |
---|---|
US (2) | US8813116B2 (en) |
EP (1) | EP2702769A4 (en) |
CN (1) | CN103493495A (en) |
WO (1) | WO2012145833A1 (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106888385A (en) * | 2017-01-17 | 2017-06-23 | 武汉噢易云计算股份有限公司 | Video broadcasting method and system of the client under virtualized environment |
US10992960B2 (en) * | 2019-02-06 | 2021-04-27 | Jared Michael Cohn | Accelerated video exportation to multiple destinations |
Families Citing this family (61)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7991910B2 (en) | 2008-11-17 | 2011-08-02 | Amazon Technologies, Inc. | Updating routing information based on client location |
US8028090B2 (en) | 2008-11-17 | 2011-09-27 | Amazon Technologies, Inc. | Request routing utilizing client location information |
US7970820B1 (en) | 2008-03-31 | 2011-06-28 | Amazon Technologies, Inc. | Locality based content distribution |
US7962597B2 (en) | 2008-03-31 | 2011-06-14 | Amazon Technologies, Inc. | Request routing based on class |
US8321568B2 (en) | 2008-03-31 | 2012-11-27 | Amazon Technologies, Inc. | Content management |
US8606996B2 (en) | 2008-03-31 | 2013-12-10 | Amazon Technologies, Inc. | Cache optimization |
US9407681B1 (en) | 2010-09-28 | 2016-08-02 | Amazon Technologies, Inc. | Latency measurement in resource requests |
US9215423B2 (en) | 2009-03-30 | 2015-12-15 | Time Warner Cable Enterprises Llc | Recommendation engine apparatus and methods |
US11076189B2 (en) | 2009-03-30 | 2021-07-27 | Time Warner Cable Enterprises Llc | Personal media channel apparatus and methods |
US8782236B1 (en) | 2009-06-16 | 2014-07-15 | Amazon Technologies, Inc. | Managing resources using resource expiration data |
US8813124B2 (en) | 2009-07-15 | 2014-08-19 | Time Warner Cable Enterprises Llc | Methods and apparatus for targeted secondary content insertion |
US8397073B1 (en) | 2009-09-04 | 2013-03-12 | Amazon Technologies, Inc. | Managing secure content in a content delivery network |
US9495338B1 (en) | 2010-01-28 | 2016-11-15 | Amazon Technologies, Inc. | Content distribution network |
US8701138B2 (en) | 2010-04-23 | 2014-04-15 | Time Warner Cable Enterprises Llc | Zone control methods and apparatus |
US9003035B1 (en) | 2010-09-28 | 2015-04-07 | Amazon Technologies, Inc. | Point of presence management in request routing |
US9712484B1 (en) | 2010-09-28 | 2017-07-18 | Amazon Technologies, Inc. | Managing request routing information utilizing client identifiers |
US10958501B1 (en) | 2010-09-28 | 2021-03-23 | Amazon Technologies, Inc. | Request routing information based on client IP groupings |
US8468247B1 (en) | 2010-09-28 | 2013-06-18 | Amazon Technologies, Inc. | Point of presence management in request routing |
US8452874B2 (en) | 2010-11-22 | 2013-05-28 | Amazon Technologies, Inc. | Request routing processing |
US10467042B1 (en) | 2011-04-27 | 2019-11-05 | Amazon Technologies, Inc. | Optimized deployment based upon customer locality |
US9426123B2 (en) | 2012-02-23 | 2016-08-23 | Time Warner Cable Enterprises Llc | Apparatus and methods for content distribution to packet-enabled devices via a network bridge |
US20130227283A1 (en) * | 2012-02-23 | 2013-08-29 | Louis Williamson | Apparatus and methods for providing content to an ip-enabled device in a content distribution network |
US10069885B2 (en) * | 2013-03-18 | 2018-09-04 | Ericsson Ab | Bandwidth management for over-the-top adaptive streaming |
US8838149B2 (en) | 2012-04-02 | 2014-09-16 | Time Warner Cable Enterprises Llc | Apparatus and methods for ensuring delivery of geographically relevant content |
US9467723B2 (en) | 2012-04-04 | 2016-10-11 | Time Warner Cable Enterprises Llc | Apparatus and methods for automated highlight reel creation in a content delivery network |
US9154551B1 (en) | 2012-06-11 | 2015-10-06 | Amazon Technologies, Inc. | Processing DNS queries to identify pre-processing information |
JP6074933B2 (en) * | 2012-07-19 | 2017-02-08 | 沖電気工業株式会社 | Video distribution apparatus, video distribution program, cache control apparatus, cache control program, video distribution system, and video distribution method |
US20140040496A1 (en) * | 2012-08-06 | 2014-02-06 | General Instrument Corporation | On-demand http stream generation |
US9124672B2 (en) * | 2012-11-08 | 2015-09-01 | Morega Systems, Inc | Adaptive video server with data rate control and methods for use therewith |
CN103856543B (en) * | 2012-12-07 | 2019-02-15 | 腾讯科技(深圳)有限公司 | A kind of method for processing video frequency, mobile terminal and server |
WO2014145011A2 (en) * | 2013-03-15 | 2014-09-18 | General Instrument Corporation | Method and apparatus for streaming video |
US9118630B2 (en) | 2013-05-14 | 2015-08-25 | Morega Systems Inc. | Client proxy for key exchange in HTTP live streaming |
US9584556B2 (en) * | 2013-05-14 | 2017-02-28 | Morega Systems Inc. | Client proxy for adaptive bitrate selection in HTTP live streaming |
WO2015124210A1 (en) * | 2014-02-21 | 2015-08-27 | Telefonaktiebolaget L M Ericsson (Publ) | Service delivery in a communication network |
US10097448B1 (en) | 2014-12-18 | 2018-10-09 | Amazon Technologies, Inc. | Routing mode and point-of-presence selection service |
US10116676B2 (en) | 2015-02-13 | 2018-10-30 | Time Warner Cable Enterprises Llc | Apparatus and methods for data collection, analysis and service modification based on online activity |
US10225326B1 (en) | 2015-03-23 | 2019-03-05 | Amazon Technologies, Inc. | Point of presence based data uploading |
US9832141B1 (en) | 2015-05-13 | 2017-11-28 | Amazon Technologies, Inc. | Routing based request correlation |
US9826261B2 (en) | 2015-09-09 | 2017-11-21 | Ericsson Ab | Fast channel change in a multicast adaptive bitrate (MABR) streaming network using multicast repeat segment bursts in a dedicated bandwidth pipe |
US9826262B2 (en) * | 2015-09-09 | 2017-11-21 | Ericsson Ab | Fast channel change in a multicast adaptive bitrate (MABR) streaming network using multicast repeat segment bursts in a shared progressive ABR download pipe |
US9917690B2 (en) * | 2015-10-01 | 2018-03-13 | Time Warner Cable Enterprises Llc | Encryption management, content recording management, and playback management in a network environment |
US10277928B1 (en) | 2015-10-06 | 2019-04-30 | Amazon Technologies, Inc. | Dynamic manifests for media content playback |
US10270878B1 (en) | 2015-11-10 | 2019-04-23 | Amazon Technologies, Inc. | Routing for origin-facing points of presence |
CN105827700A (en) * | 2016-03-15 | 2016-08-03 | 北京金山安全软件有限公司 | Dynamic file transmission method and device and electronic equipment |
US10075292B2 (en) * | 2016-03-30 | 2018-09-11 | Divx, Llc | Systems and methods for quick start-up of playback |
US10075551B1 (en) | 2016-06-06 | 2018-09-11 | Amazon Technologies, Inc. | Request management for hierarchical cache |
US10110694B1 (en) * | 2016-06-29 | 2018-10-23 | Amazon Technologies, Inc. | Adaptive transfer rate for retrieving content from a server |
US11212593B2 (en) | 2016-09-27 | 2021-12-28 | Time Warner Cable Enterprises Llc | Apparatus and methods for automated secondary content management in a digital network |
US10616250B2 (en) | 2016-10-05 | 2020-04-07 | Amazon Technologies, Inc. | Network addresses with encoded DNS-level information |
US20180152737A1 (en) * | 2016-11-28 | 2018-05-31 | Facebook, Inc. | Systems and methods for management of multiple streams in a broadcast |
US10831549B1 (en) | 2016-12-27 | 2020-11-10 | Amazon Technologies, Inc. | Multi-region request-driven code execution system |
US10938884B1 (en) | 2017-01-30 | 2021-03-02 | Amazon Technologies, Inc. | Origin server cloaking using virtual private cloud network environments |
US10771855B1 (en) * | 2017-04-10 | 2020-09-08 | Amazon Technologies, Inc. | Deep characterization of content playback systems |
US11075987B1 (en) | 2017-06-12 | 2021-07-27 | Amazon Technologies, Inc. | Load estimating content delivery network |
JP6797755B2 (en) * | 2017-06-20 | 2020-12-09 | キヤノン株式会社 | Imaging device, processing method and program of imaging device |
US10742593B1 (en) | 2017-09-25 | 2020-08-11 | Amazon Technologies, Inc. | Hybrid content request routing system |
US10440367B1 (en) | 2018-06-04 | 2019-10-08 | Fubotv Inc. | Systems and methods for adaptively encoding video stream |
US10862852B1 (en) | 2018-11-16 | 2020-12-08 | Amazon Technologies, Inc. | Resolution of domain name requests in heterogeneous network environments |
US11025747B1 (en) | 2018-12-12 | 2021-06-01 | Amazon Technologies, Inc. | Content request pattern-based routing system |
US11910056B2 (en) * | 2020-09-24 | 2024-02-20 | Centurylink Intellectual Property Llc | Content delivery using distributed ledger and AI-based transcoding technologies |
US11962825B1 (en) | 2022-09-27 | 2024-04-16 | Amazon Technologies, Inc. | Content adjustment system for reduced latency |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7127493B1 (en) * | 1998-08-20 | 2006-10-24 | Gautier Taylor S | Optimizing server delivery of content by selective inclusion of optional data based on optimization criteria |
US20080120676A1 (en) * | 2006-11-22 | 2008-05-22 | Horizon Semiconductors Ltd. | Integrated circuit, an encoder/decoder architecture, and a method for processing a media stream |
US20110231660A1 (en) * | 2010-03-22 | 2011-09-22 | Echostar Technologies Llc | Systems and methods for securely streaming media content |
US20110238789A1 (en) * | 2006-06-09 | 2011-09-29 | Qualcomm Incorporated | Enhanced block-request streaming system using signaling or block creation |
Family Cites Families (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7203758B2 (en) * | 2000-10-19 | 2007-04-10 | Loudeye Technologies, Inc. | System and method for selective insertion of content into streaming media |
GB0230328D0 (en) * | 2002-12-31 | 2003-02-05 | British Telecomm | Video streaming |
US7818444B2 (en) * | 2004-04-30 | 2010-10-19 | Move Networks, Inc. | Apparatus, system, and method for multi-bitrate content streaming |
US8099755B2 (en) * | 2004-06-07 | 2012-01-17 | Sling Media Pvt. Ltd. | Systems and methods for controlling the encoding of a media stream |
US8073275B2 (en) | 2005-02-09 | 2011-12-06 | Mobixell Networks Ltd. | Image adaptation with target size, quality and resolution constraints |
US8218657B2 (en) * | 2005-09-02 | 2012-07-10 | Netgear, Inc. | System and method for automatic adjustment of streaming video bit rate |
US20070116117A1 (en) * | 2005-11-18 | 2007-05-24 | Apple Computer, Inc. | Controlling buffer states in video compression coding to enable editing and distributed encoding |
US7889765B2 (en) * | 2005-11-30 | 2011-02-15 | Time Warner Cable Inc. | Apparatus and methods for utilizing variable rate program streams in a network |
US8711929B2 (en) * | 2006-11-01 | 2014-04-29 | Skyfire Labs, Inc. | Network-based dynamic encoding |
WO2009045189A1 (en) * | 2007-10-02 | 2009-04-09 | Alcatel Lucent | Method and system for facilitating track-based server-side streaming playlist functionality |
US20110124986A1 (en) | 2008-07-23 | 2011-05-26 | Masaaki Takama | Microchip, Blood Analysis System and Blood Analysis Method |
US9060187B2 (en) | 2008-12-22 | 2015-06-16 | Netflix, Inc. | Bit rate stream switching |
US8099476B2 (en) * | 2008-12-31 | 2012-01-17 | Apple Inc. | Updatable real-time or near real-time streaming |
CA2755774C (en) | 2009-03-19 | 2015-01-06 | Azuki Systems, Inc. | Method for scalable live streaming delivery for mobile audiences |
JP4984183B2 (en) * | 2009-04-07 | 2012-07-25 | ソニー株式会社 | REPRODUCTION DEVICE, REPRODUCTION METHOD, AND RECORDING METHOD |
US20110066703A1 (en) * | 2009-05-20 | 2011-03-17 | Creative Ad Technology Proprietary Limited | Methods and systems for delivering media to client device |
US8954596B2 (en) * | 2010-04-02 | 2015-02-10 | Netflix, Inc. | Dynamic virtual chunking of streaming media content |
-
2011
- 2011-08-03 US US13/197,400 patent/US8813116B2/en active Active
-
2012
- 2012-04-23 WO PCT/CA2012/000399 patent/WO2012145833A1/en active Application Filing
- 2012-04-23 EP EP12776412.4A patent/EP2702769A4/en not_active Withdrawn
- 2012-04-23 CN CN201280020197.4A patent/CN103493495A/en active Pending
-
2014
- 2014-05-21 US US14/284,195 patent/US20140331262A1/en not_active Abandoned
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7127493B1 (en) * | 1998-08-20 | 2006-10-24 | Gautier Taylor S | Optimizing server delivery of content by selective inclusion of optional data based on optimization criteria |
US20110238789A1 (en) * | 2006-06-09 | 2011-09-29 | Qualcomm Incorporated | Enhanced block-request streaming system using signaling or block creation |
US20080120676A1 (en) * | 2006-11-22 | 2008-05-22 | Horizon Semiconductors Ltd. | Integrated circuit, an encoder/decoder architecture, and a method for processing a media stream |
US20110231660A1 (en) * | 2010-03-22 | 2011-09-22 | Echostar Technologies Llc | Systems and methods for securely streaming media content |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106888385A (en) * | 2017-01-17 | 2017-06-23 | 武汉噢易云计算股份有限公司 | Video broadcasting method and system of the client under virtualized environment |
US10992960B2 (en) * | 2019-02-06 | 2021-04-27 | Jared Michael Cohn | Accelerated video exportation to multiple destinations |
US11463732B2 (en) | 2019-02-06 | 2022-10-04 | Evergreen Groupe, Llc | Accelerated video exportation to multiple destinations |
US11503340B2 (en) | 2019-02-06 | 2022-11-15 | Evergreen Groupe, Llc | Accelerated video exportation to multiple destinations |
Also Published As
Publication number | Publication date |
---|---|
US8813116B2 (en) | 2014-08-19 |
EP2702769A4 (en) | 2014-09-17 |
US20120278833A1 (en) | 2012-11-01 |
CN103493495A (en) | 2014-01-01 |
WO2012145833A1 (en) | 2012-11-01 |
EP2702769A1 (en) | 2014-03-05 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8813116B2 (en) | Adaptive video server with virtual file system and methods for use therewith | |
US9167285B2 (en) | Streaming video server with virtual file system and methods for use therewith | |
US9124672B2 (en) | Adaptive video server with data rate control and methods for use therewith | |
US8918821B2 (en) | Client device with video playlist translation via client-side proxy and methods for use therewith | |
US9015779B2 (en) | Streaming video server with segment length control and methods for use therewith | |
US9325945B2 (en) | Video server and client with custom key exchange and methods for use therewith | |
US9351020B2 (en) | On the fly transcoding of video on demand content for adaptive streaming | |
US9661051B2 (en) | Streamloading content, such as video content for example, by both downloading enhancement layers of the content and streaming a base layer of the content | |
US8627350B2 (en) | Systems and method for determining visual media information | |
JP2022503419A (en) | Systems and methods for real-time adaptive bitrate transcoding and transmission of transcoded media | |
US20190174159A1 (en) | Adaptive video consumption | |
US9294821B2 (en) | Scrubbing noise remover and methods for use therewith | |
US11805290B2 (en) | Method for managing zapping of digital multimedia contents obtained by HTTP adaptive streaming (HAS), and corresponding management device, multimedia stream reader and computer program | |
US20140282767A1 (en) | Asymmetric content delivery of media content | |
JP2018074349A (en) | Video processing device, video processing method and video processing program | |
US20210211485A1 (en) | Method for managing progressive and adaptive streaming of digital content by a multimedia stream player terminal connected to a communication network, and associated management device, multimedia stream player terminal and computer program |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |
|
AS | Assignment |
Owner name: MOREGA SYSTEMS INC., CANADA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:TAM, KING CHIU;REEL/FRAME:049495/0208 Effective date: 20110811 |
|
AS | Assignment |
Owner name: MOREGA SYSTEMS INC., CANADA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:TAM, KING CHIU;REEL/FRAME:048718/0768 Effective date: 20110811 |
|
AS | Assignment |
Owner name: 3311148 NOVA SCOTIA LIMITED, CANADA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MOREGA SYSTEMS INC.;REEL/FRAME:048746/0705 Effective date: 20171215 |
|
AS | Assignment |
Owner name: AT&T INTELLECTUAL PROPERTY I, L.P., GEORGIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:3311148 NOVA SCOTIA LIMITED;REEL/FRAME:048754/0219 Effective date: 20190114 |