Disclosed is an audio signal encoding/decoding method that uses an encoding downmix strategy applied at an encoder that is different than a decoding re-mix/upmix strategy applied at a decoder. Based on the type of downmix coding scheme, the method comprises: computing input downmixing gains to be applied to the input audio signal to construct a primary downmix channel; determining downmix scaling gains to scale the primary downmix channel; generating prediction gains based on the input audio signal, the input downmixing gains and the downmix scaling gains; determining residual channel(s) from the side channels by using the primary downmix channel and the prediction gains to generate side channel predictions and subtracting the side channel predictions from the side channels; determining decorrelation gains based on energy in the residual channels; encoding the primary downmix channel, the residual channel(s), the prediction gains and the decorrelation gains; and sending the bitstream to a decoder.
G10L 19/008 - Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
G10L 19/083 - Determination or coding of the excitation function; Determination or coding of the long-term prediction parameters the excitation function being an excitation gain
H04S 7/00 - Indicating arrangements; Control arrangements, e.g. balance control
2.
END-TO-END GENERAL AUDIO SYNTHESIS WITH GENERATIVE NETWORKS
An aspect of the present disclosure relates to a neural network-based system for general audio synthesis comprising a generator configured to generate synthesized audio. The generator comprising an encoder configured to transform an input audio signal with a first rate into a sequence of hidden features with a second rate, lower than the first rate and process the hidden features to aggregate temporal information. The generator comprises a decoder configured to convert the hidden features back to the first rate by upsampling to form a processed signal and output a synthesized audio signal based on the processed signal as the generated synthesized audio.
G10L 21/045 - Time compression or expansion by changing speed using thinning out or insertion of a waveform
G10L 25/30 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the analysis technique using neural networks
G10L 13/02 - Methods for producing synthetic speech; Speech synthesisers
G10L 21/00 - Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
3.
METHODS, APPARATUS AND SYSTEMS FOR ENCODING AND DECODING OF MULTI-CHANNEL AMBISONICS AUDIO DATA
Conventional audio compression technologies perform a standardized signal transformation, independent of the type of the content. Multi-channel signals are decomposed into their signal components, subsequently quantized and encoded. This is disadvantageous due to lack of knowledge on the characteristics of scene composition, especially for e.g. multi-channel audio or Higher-Order Ambisonics (HOA) content. A method for decoding an encoded bitstream of multi-channel audio data and associated metadata is provided, including transforming the first Ambisonics format of the multi-channel audio data to a second Ambisonics format representation of the multi-channel audio data, wherein the transforming maps the first Ambisonics format of the multi-channel audio data into the second Ambisonics format representation of the multi-channel audio data. A method for encoding multi-channel audio data that includes audio data in an Ambisonics format, wherein the encoding includes transforming the audio data in an Ambisonics format into encoded multi-channel audio data is also provided.
G10L 19/008 - Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
H04S 3/00 - Systems employing more than two channels, e.g. quadraphonic
4.
CONVERSION OF SCENE BASED AUDIO REPRESENTATIONS TO OBJECT BASED AUDIO REPRESENTATIONS
A mixing matrix, suitable for converting a scene-based audio (SBA) input signal to an object-based audio (OBA) signal, is constructed so that the resulting OBA signal is composed of object signals with amplitudes that are biased according to amplitude preference coefficients. The amplitude preference coefficients are chosen to place dominant spatial audio objects in a fewer number of output object channels, to provide a more discrete OBA rendering of the SBA input signal.
H04S 3/02 - Systems employing more than two channels, e.g. quadraphonic of the matrix type, i.e. in which input signals are combined algebraically, e.g. after having been phase shifted with respect to each other
Methods and systems for canvas size scalability across the same or different bitstream layers of a video coded bitstream are described. Offset parameters for a conformance window, a reference region of interest (ROI) in a reference layer, and a current ROI in a current layer are received. The width and height of a current ROI and a reference ROI are computed based on the offset parameters and they are used to generate a width and height scaling factor to be used by a reference picture resampling unit to generate an output picture based on the current ROI and the reference ROI.
H04N 19/105 - Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
H04N 19/172 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
H04N 19/33 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability in the spatial domain
6.
METHOD, APPARATUS, AND MEDIUM FOR ENCODING AND DECODING OF AUDIO BITSTREAMS AND ASSOCIATED RETURN CHANNEL INFORMATION
A method, performed by a device with one or more microphones, for generating an encoded bitstream, the method comprising, capturing, by the one or more microphones, one or more audio signals, analyzing the captured audio signals to determine presence of a wake word, upon detecting presence of a wake word, setting a flag to indicate a speech recognition task is to be performed on the captured audio signals, encoding the captured audio signals, assembling the encoded audio signals and the flag into the encoded bitstream.
G10L 15/22 - Procedures used during a speech recognition process, e.g. man-machine dialog
G10L 15/30 - Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
G10L 25/18 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the type of extracted parameters the extracted parameters being spectral information of each sub-band
G10L 25/24 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the type of extracted parameters the extracted parameters being the cepstrum
7.
METHODS, APPARATUS AND SYSTEMS FOR PERFORMING PERCEPTUALLY MOTIVATED GAIN CONTROL
Systems, methods, and computer program products for performing gain control on audio signals are provided. An automatic gain control system obtains a downmixed audio signal of an audio signal to be encoded. The system determines that an overload condition has occurred for a frame of the downmixed audio signal. Responsive to the overload condition, the system determines a gain transition function for the frame, wherein the gain transition function is based at least on a gain transition step size. The system applies the gain transition function to the frame to generate a gain adjusted frame of the downmixed audio signal. The system provides the gain adjusted frame and information indicative of the gain transition function for encoding by an encoder.
G10L 19/008 - Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
8.
METHOD, APPARATUS, AND MEDIUM FOR ENCODING AND DECODING OF AUDIO BITSTREAMS WITH PARAMETRIC FLEXIBLE RENDERING CONFIGURATION DATA
A method for generating an encoded bitstream from an audio program comprising a plurality of audio signals, the method comprising receiving, for each of the plurality of audio signals, information indicating a playback device with which the respective audio signal is associated, receiving, for each playback device, information indicating at least one of a delay, a gain, and an equalization curve associated with the respective playback device, determining, from the plurality of audio signals, a group of two or more related audio signals, applying one or more joint-coding tools to the two or more related audio signals of the group to obtain jointly-coded audio signals, and combining the jointly-coded audio signals, an indication of the playback devices with which the jointly-coded audio signals are associated, and indications of the delay and the gain associated with the respective playback devices with which the jointly-coded audio signals are associated, into an independent block of an encoded bitstream.
G10L 19/008 - Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
A method for generating a frame of an encoded bitstream of an audio program comprising a plurality of audio signals, wherein the frame comprises two or more independent blocks of encoded data, the method comprising receiving, for one or more of the plurality of audio signals, information indicating a playback device with which the one or more audio signals are associated, receiving, for the indicated playback device, information indicating one or more additional associated playback devices, receiving one or more audio signals associated with the indicated one or more additional associated playback devices, encoding the one or more audio signals associated with the playback device, encoding the one or more audio signals associated with the indicated one or more additional associated playback devices, combining the one or more encoded audio signals associated with the playback device and signaling information indicating the one or more additional associated playback devices into a first independent block, combining the one or more encoded audio signals associated with the one or more additional associated playback devices into one or more additional independent blocks, and combining the first independent block and the one or more additional independent blocks into the frame of the encoded bitstream.
G10L 19/008 - Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
Methods, systems, and devices implement intra-prediction for hexagonally-sampled compression and decompression of videos and images having a regular grid of hexagonally-shaped pixels. For encoding, a prediction unit (PU) shape is selected at a sequence level from the group consisting of parallelogram, zigzag-square, hexagonal super-pixel, a rectangular zigzag and an arrow, and the hexagonally-sampled image is divided into regions based on the PU shape. For each region: a prediction mode and a PU size are determined; reference pixels are determined for each predicted pixel in the PU shape based on the prediction mode; a weighted factor is determined for each of the reference pixels based on a distance between the reference pixel and the predicted pixel; and a predicted value of each of the predicted pixels in the PU shape is determined using the corresponding reference pixels and the weighted factors.
H04N 19/105 - Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
H04N 19/119 - Adaptive subdivision aspects e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
H04N 19/159 - Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
H04N 19/176 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
11.
REPRESENTING SPATIAL AUDIO BY MEANS OF AN AUDIO SIGNAL AND ASSOCIATED METADATA
There is provided encoding and decoding methods for representing spatial audio that is a combination of directional sound and diffuse sound. An exemplary encoding method includes inter alia creating a single- or multi-channel downmix audio signal by downmixing input audio signals from a plurality of microphones in an audio capture unit capturing the spatial audio; determining first metadata parameters associated with the downmix audio signal, wherein the first metadata parameters are indicative of one or more of: a relative time delay value, a gain value, and a phase value associated with each input audio signal; and combining the created downmix audio signal and the first metadata parameters into a representation of the spatial audio.
H04S 3/02 - Systems employing more than two channels, e.g. quadraphonic of the matrix type, i.e. in which input signals are combined algebraically, e.g. after having been phase shifted with respect to each other
12.
FREQUENCY DOMAIN MULTIPLEXING OF SPATIAL AUDIO FOR MULTIPLE LISTENER SWEET SPOTS
Some methods involve receiving, by a control system that is configured for implementing a plurality of renderers, audio data and listening configuration data for a plurality of listening configurations, each listening configuration of the plurality of listening configurations corresponding to a listening position and a listening orientation in an audio environment, and rendering, by each renderer and according to the listening configuration data, the received audio data to obtain a set of renderer-specific loudspeaker feed signals for a corresponding listening configuration. Each renderer may be configured to render the audio data for a different listening configuration. Some such methods may involve decomposing each set of renderer-specific loudspeaker feed signals into a renderer-specific set of frequency bands and combining the renderer-specific frequency bands of each renderer to produce an output set of loudspeaker feed signals.
Some examples involve rendering received audio data by determining a first relative activation of a set of loudspeakers in an environment according to a first rendering configuration corresponding to a first set of speaker activations, receiving a first rendering transition indication indicating a transition from the first rendering configuration to a second rendering configuration and determining a second set of speaker activations corresponding to a simplified version of the second rendering configuration. Some examples involve performing a first transition from the first set of speaker activations to the second set of speaker activations, determining a third set of speaker activations corresponding to a complete version of the second rendering configuration and performing a second transition to the third set of speaker activations without requiring completion of the first transition.
A first image and a second image of different dynamic ranges are derived from the same source image. Based on a chroma sampling format of the first image, it is determined whether edge preserving filtering is to be used to generate chroma upsampled image data in a reconstructed image. If so, image metadata for performing the edge preserving filtering is generated. The first image, the second image and the image metadata are encoded into an image data container to enable a recipient device to generate the reconstructed image.
H04N 19/184 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being bits, e.g. of the compressed video stream
G06V 10/25 - Determination of region of interest [ROI] or a volume of interest [VOI]
H04N 19/117 - Filters, e.g. for pre-processing or post-processing
H04N 19/172 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
H04N 19/59 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
An multi-input, multi-output audio process is implemented as a linear system for use in an audio filterbank to convert a set of frequency-domain input audio signals into a set of frequency-domain output audio signals. A transfer function from one input to one output is defined as a frequency dependent gain function. In some implementations, the transfer function includes a direct component that is substantially defined as a frequency dependent gain, and one or more decorrelated components that have frequency-varying group phase response. The transfer function is formed from a set of sub-band functions, with each sub-band function being formed from a set of corresponding component transfer functions including direct component and one or more decorrelated components.
H04S 3/02 - Systems employing more than two channels, e.g. quadraphonic of the matrix type, i.e. in which input signals are combined algebraically, e.g. after having been phase shifted with respect to each other
H04S 5/00 - Pseudo-stereo systems, e.g. in which additional channel signals are derived from monophonic signals by means of phase shifting, time delay or reverberation
16.
NEURAL SEGMENTATION FIELDS FOR REPRESENTING THREE-DIMENSIONAL SCENES
Methods and apparatus for rendering segmentation maps of a 3D scene using machine learning. According to an example embodiment, a method of training a neural network to render segmentation maps corresponding to arbitrarily selected views of a 3D scene comprises: computing color texture and volume density corresponding to a selected training view of the 3D scene, the computing being performed using a 3D representation pretrained to represent the 3D scene; generating a predicted segmentation map corresponding to the selected training view of the 3D scene, the generating being performed using the neural network based on said color texture and said volume density; and adjusting configuration parameters of network nodes of the neural network based on a loss function configured to receive, as a first input thereof, a ground-truth segmentation map corresponding to the selected training view and further configured to receive, as a second input thereof, the predicted segmentation map.
Some methods involve receiving, by a control system configured for implementing a plurality of Tenderers, audio data and listening configuration data for a plurality of listening configurations, each listening configuration of the plurality of listening configurations corresponding to a listening position and a listening orientation in an audio environment, and rendering, by each Tenderer and according to the listening configuration data, the received audio data to obtain a set of Tenderer-specific loudspeaker feed signals for a corresponding listening configuration. Each Tenderer may be configured to render the audio data for a different listening configuration. Some such methods may involve decomposing each set of renderer-specific loudspeaker feed signals into a Tenderer-specific set of frequency bands and combining the renderer-specific frequency bands of each Tenderer to produce an output set of loudspeaker feed signals. Some such methods may involve outputting the output set of loudspeaker feed signals to a plurality of loudspeakers.
Embodiments are disclosed for spatial noise filling in multi-channel codecs. In an embodiment, a method of regenerating background noise ambience in a multi-channel codec by generating spatial hole filling noise comprises: computing noise estimates based on a primary downmix channel generated from an input audio signal representing a spatial audio scene with background noise ambience; computing spectral shaping filter coefficients based on the noise estimates; spectrally shaping the multi-channel noise signal using the spectral shaping filter coefficients and a noise distribution, the spectral shaping resulting in a diffused, multi-channel noise signal with uncorrelated channels; spatially shaping the diffused, uncorrelated multi-channel noise signal with uncorrelated channels based on a noise ambience of the spatial audio scene; and adding the spatially and spectrally shaped multi-channel noise to a multi-channel codec output to synthesize the background noise ambience of the spatial audio scene.
G10L 19/03 - Spectral prediction for preventing pre-echo; Temporary noise shaping [TNS], e.g. in MPEG2 or MPEG4
G10L 19/008 - Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
G10L 21/0216 - Noise filtering characterised by the method used for estimating noise
19.
SINGLE CHANNEL ENCODING INTO A MULTI-CHANNEL CONTAINER FOLLOWED BY IMAGE COMPRESSION
Coding methods and apparatus for packing single-channel data into a multi-channel container, e.g., an MP4, TIFF, or JPEG container, to at least achieve good utilization of the container's data capacity. In some examples, a coding method comprises: converting a plurality of scalar values of a received data stream into a corresponding plurality of n-dimensional values, the converting being performed using a mapper; assigning each of the n-dimensional values as a pixel value to a respective pixel of a virtual-image frame, where n is an integer greater than one; and compressing the virtual-image frame according to a type of a container for image data. The mapper is configured to map a scalar value to a corresponding n-dimensional value based on a relationship represented by an n-dimensional curve or by a plurality of 2n-way tree partitions of n-dimensional space.
H04N 19/85 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
H04N 19/88 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving rearrangement of data among different coding units, e.g. shuffling, interleaving, scrambling or permutation of pixel data or permutation of transform coefficient data among different blocks
H04N 19/90 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups , e.g. fractals
Embodiments are directed to a method and system for receiving, in a bitstream, metadata associated with the audio data, and analyzing the metadata to determine whether a loudness parameter for a first group of audio playback devices are available in the bitstream. Responsive to determining that the parameters are present for the first group, the system uses the parameters and audio data to render audio. Responsive to determining that the loudness parameters are not present for the first group, the system analyzes one or more characteristics of the first group, and determines the parameter based on the one or more characteristics.
A method for encoding an input audio stream including the steps of obtaining a first playback stream presentation of the input audio stream intended for reproduction on a first audio reproduction system, obtaining a second playback stream presentation of the input audio stream intended for reproduction on a second audio reproduction system, determining a set of transform parameters suitable for transforming an intermediate playback stream presentation to an approximation of the second playback stream presentation, wherein the transform parameters are determined by minimization of a measure of a difference between the approximation of the second playback stream presentation and the second playback stream presentation, and encoding the first playback stream presentation and the set of transform parameters for transmission to a decoder.
G10L 19/008 - Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
An attenuation or “gap” may be inserted into at least a first frequency range of at least first and second audio playback signals of a content stream during at least a first time interval to generate at least first and second modified audio playback signals. Corresponding audio device playback sound may be provided by at least first and second audio devices. At least one microphone may detect at least the first audio device playback sound and the second audio device playback sound and may generate corresponding microphone signals. Audio data may be extracted from the microphone signals in at least the first frequency range, to produce extracted audio data. A far-field audio environment impulse response and/or audio environment noise may be estimated based, at least in part, on the extracted audio data.
A first reshaping mapping is performed on a first image represented in a first domain to generate a second image represented in a second domain. The first domain is of a first dynamic range different from a second dynamic range of which the second domain is. A second reshaping mapping is performed on the second image represented in the second domain to generate a third image represented in the first domain. The third image is perceptually different from the first image in at least one of: global contrast, global saturation, local contrast, local saturation, etc. A display image is derived from the third image and rendered on a display device.
A method for performing DRC on a HOA signal comprises transforming the HOA signal to the spatial domain, analyzing the transformed HOA signal, and obtaining, from results of said analyzing, gain factors that are usable for dynamic compression. The gain factors can be transmitted together with the HOA signal. When applying the DRC, the HOA signal is transformed to the spatial domain, the gain factors are extracted and multiplied with the transformed HOA signal in the spatial domain, wherein a gain compensated transformed HOA signal is obtained. The gain compensated transformed HOA signal is transformed back into the HOA domain, wherein a gain compensated HOA signal is obtained. The DRC may be applied in the QMF-filter bank domain.
H04S 3/00 - Systems employing more than two channels, e.g. quadraphonic
G10L 19/008 - Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
H04S 3/02 - Systems employing more than two channels, e.g. quadraphonic of the matrix type, i.e. in which input signals are combined algebraically, e.g. after having been phase shifted with respect to each other
25.
SYSTEMS AND METHODS FOR LOCAL DIMMING IN MULTI-MODULATION DISPLAYS
Dual and multi-modulator projector display systems and techniques are disclosed. In one embodiment, a projector display system comprises a light source; a controller, a first modulator, receiving light from the light source and rendering a halftone image of said the input image; a blurring optical system that blurs said halftone image with a Point Spread Function (PSF); and a second modulator receiving the blurred halftone image and rendering a pulse width modulated image which may be projected to form the desired screen image. Systems and techniques for forming a binary halftone image from input image, correcting for misalignment between the first and second modulators and calibrating the projector system—e.g. over time—for continuous image improvement are also disclosed.
H04N 9/31 - Projection devices for colour picture display
B65B 11/04 - Wrapping articles or quantities of material, without changing their position during the wrapping operation, e.g. in moulds with hinged folders the articles being rotated
B65B 11/48 - Enclosing articles, or quantities of material, by folding a wrapper, e.g. a pocketed wrapper, and securing its opposed free margins to enclose contents
B65B 11/58 - Applying two or more wrappers, e.g. in succession
B65B 55/00 - Preserving, protecting or purifying packages or package contents in association with packaging
B65B 61/06 - Auxiliary devices, not otherwise provided for, for operating on sheets, blanks, webs, binding material, containers or packages for severing webs, or for separating joined packages by cutting
B65B 61/26 - Auxiliary devices, not otherwise provided for, for operating on sheets, blanks, webs, binding material, containers or packages for marking or coding completed packages
G03B 21/00 - Projectors or projection-type viewers; Accessories therefor
G03B 21/13 - Projectors for producing special effects at the edges of picture, e.g. blurring
Images are acquired through image sensors operating in conjunction with a media consumption system. The acquired images are used to determine a user's movement in a plurality of degrees of freedom. Sound images depicted in spatial audio rendered by audio speakers operating in conjunction with the media consumption system are adapted based at least in part on the user's movement in the plurality of degrees of freedom.
A computer implemented system for rendering captured audio soundfields to a listener comprises apparatus to deliver the audio soundfields to the listener. The delivery apparatus delivers the audio soundfields to the listener with first and second audio elements perceived by the listener as emanating from first and second virtual source locations, respectively, and with the first audio element and/or the second audio element delivered to the listener from a third virtual source location. The first virtual source location and the second virtual source location are perceived by the listener as being located to the front of the listener, and the third virtual source location is located to the rear or the side of the listener.
The present invention is directed to methods and apparatus for translating a first plurality of audio input channels to a second plurality of audio output channels. This includes determining that there is pair-wise coding among any of the first plurality of audio input channels, determining an input/output-mapping matrix for mapping at least a first set of the first plurality of audio input channels to at least a second set of the second plurality of audio output channels; and deriving the second plurality of audio output channels based on first plurality of audio input channels, the input/output-mapping matrix and the determined pair-wise coding. The first plurality of audio input channels represent the same soundfield represented by the second plurality of audio output channels.
H04S 5/00 - Pseudo-stereo systems, e.g. in which additional channel signals are derived from monophonic signals by means of phase shifting, time delay or reverberation
29.
VIDEO CODING METHOD AND APPARATUS USING ANY TYPES OF BLOCK PARTITIONING
The present invention relates to a block partitioning structure in video coding technology, and a video encoding and decoding method and apparatus using the same, wherein the video encoding and decoding method includes the steps of: acquiring quad-partitioning information of a block; acquiring bi-partitioning information of the block when the acquired quad-partitioning information of the block does not indicate four partitions; acquiring partitioning direction information for bi-partitioning of the block when the acquired bi-partitioning information of the block indicates two partitions; acquiring information on whether to perform any other type of partitioning, when the acquired bi-partitioning information of the block does not indicate two partitions; and acquiring additional information required for the any other type of partitioning, when the acquired information on whether to perform any other type of partitioning indicates that the any other type of partitioning is performed.
H04N 19/119 - Adaptive subdivision aspects e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
H04N 19/176 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
H04N 19/46 - Embedding additional information in the video signal during the compression process
H04N 19/66 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using error resilience involving data partitioning, i.e. separation of data into packets or partitions according to importance
30.
HEAD-TRACKED SPLIT RENDERING AND HEAD-RELATED TRANSFER FUNCTION PERSONALIZATION
Systems, methods and computer program products for direction of arrival (DOA) based head-tracked split rendering and head-related transfer function (HRTF) personalization are described. Head-tracked audio rendering is split between two devices. A first device receives a main bitstream representation of encoded audio. A second device tracks head pose information. The first device decodes the main bitstream using a main decoder, and encodes the decoded bitstream into pre-rendered binaural signals and post-render metadata. The second device decodes the pre-rendered binaural signals and post-renderer metadata from the intermediate bitstream, and provides the decoded pre-rendered binaural signals and post-renderer metadata to a lightweight renderer. The lightweight renderer renders the pre-rendered binaural signals into binaural audio based on the post-renderer metadata, the head pose information, generic HRTF, and personalized HRTF.
Methods, systems, and media for utilizing head tracking data are provided. In some embodiments, a method involves receiving, at each earbud of a pair of communicatively coupled earbuds, sensor data from one or more sensors. The method may involve determining, at each earbud of the pair of communicatively coupled earbuds, head orientation information. The method may involve transmitting the determined head orientation information between the pair of communicatively coupled earbuds such that a leader earbud transmits head orientation information determined by the leader earbud to a follower earbud. The method may involve synchronizing, at each earbud, the determined head orientation data based at least in part on timing information associated with a timestamp at which the head orientation information was transmitted. The method may involve utilizing the synchronized head orientation data to present audio content by each earbud of the pair of communicatively coupled earbuds.
A system and method for the generation of automatic audio-visual analytics for object rendering in capture. One example provides a method of processing audiovisual content. The method includes receiving content including a plurality of audio frames and a plurality of video frames, classifying each of the plurality of audio frames into a plurality of audio classifications, and classifying each of the plurality of video frames into a plurality of video classifications. The method includes processing the plurality of audio frames based on the respective audio classifications and processing the plurality of video frames based on the respective video classifications. Each audio classification is processed with a different audio processing operation, and each video classification is processed with a different video processing operation. The method includes generating an audio/video representation of the content by merging the processed plurality of audio frames and the processed plurality of video frames.
H04N 21/233 - Processing of audio elementary streams
H04N 5/14 - Picture signal circuitry for video frequency region
H04N 21/234 - Processing of video elementary streams, e.g. splicing of video streams or manipulating MPEG-4 scene graphs
H04N 21/439 - Processing of audio elementary streams
H04N 21/44 - Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to MPEG-4 scene graphs
A loudspeaker assembly including a wedge-shaped acoustic waveguide and an audio driver. The wedge-shaped acoustic waveguide includes a first face including a slot opening formed therein, a second face joined at an edge with the first face, the second face tilted relative to the first face by a first angle, and a surface disposed opposite the edge and connected between the first face and the second face. The audio driver is coupled to the second face such that the audio driver is tilted at the first angle relative to the first face. The first face is oriented to face in a first direction and the audio driver is oriented to face in a second direction. A pressure level of sound energy emitted by the audio driver is greater along the second direction than the first direction. A vehicle comprising: a passenger cabin, a pillar extending upwards at a first angle relative to the passenger cabin, and the loudspeaker assembly installed on the pillar.
H04R 1/34 - Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only by using a single transducer with sound reflecting, diffracting, directing or guiding means
B60R 11/02 - Arrangements for holding or mounting articles, not otherwise provided for for radio sets, television sets, telephones, or the like; Arrangement of controls thereof
Methods and systems for frame rate scalability are described. Support is provided for input and output video sequences with variable frame rate and variable shutter angle across scenes, or for input video sequences with fixed input frame rate and input shutter angle, but allowing a decoder to generate a video output at a different output frame rate and shutter angle than the corresponding input values. Techniques allowing a decoder to decode more computationally-efficiently a specific backward compatible target frame rate and shutter angle among those allowed are also presented.
H04N 19/172 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
H04N 19/187 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a scalable video layer
H04N 19/30 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
H04N 19/31 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability in the temporal domain
H04N 19/46 - Embedding additional information in the video signal during the compression process
H04N 19/70 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
35.
IMAGE ENCODING AND DECODING APPARATUS, AND IMAGE ENCODING AND DECODING METHOD
According to the present invention, an adaptive scheme is applied to an image encoding apparatus that includes an inter-predictor, an intra-predictor, a transformer, a quantizer, an inverse quantizer, and an inverse transformer, wherein input images are classified into two or more different categories, and two or more modules from among the inter-predictor, the intra-predictor, the transformer, the quantizer, and the inverse quantizer are implemented to perform respective operations in different schemes according to the category to which an input image belongs. Thus, the invention has the advantage of efficiently encoding an image without the loss of important information as compared to a conventional image encoding apparatus which adopts a packaged scheme.
H04N 19/11 - Selection of coding mode or of prediction mode among a plurality of spatial predictive coding modes
H04N 19/117 - Filters, e.g. for pre-processing or post-processing
H04N 19/12 - Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
H04N 19/136 - Incoming video signal characteristics or properties
H04N 19/14 - Coding unit complexity, e.g. amount of activity or edge presence estimation
H04N 19/159 - Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
H04N 19/176 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
H04N 19/61 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
A projection system for pixel shifting comprising a light source configured to emit light and a spatial light modulator configured to receive the light and generate a modulated light. The spatial light modulator includes a plurality of micromirrors. The projection system includes a wobulation device configured to shift the modulated light by fractional pixels. The projection system includes a controller configured to, for each of a plurality of subperiods, control the light source to emit the light onto the spatial light modulator, and between each of the plurality of subperiods and with the wobulation device, shift the modulated light by a partial pixel distance greater than a half-pixel distance.
H04N 9/31 - Projection devices for colour picture display
H04N 5/74 - Projection arrangements for image reproduction, e.g. using eidophor
G02B 26/08 - Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light
G03B 21/00 - Projectors or projection-type viewers; Accessories therefor
G09G 3/00 - Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes
Methods, systems, and bitstream syntax are described for a scalable 3D scene representation. A general framework presents a dual-layer architecture where a base layer provides a baseline scene representation, and an enhancement layer provides enhancement information under a variety of scalability criteria. The enhancement information is coded using a trained neural field. Example systems are provided using a PSNR criterion and a baseline multi-plane image (MPI) representation. Examples of bitstream syntax for metadata information are also provided.
H04N 19/30 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
H04N 19/46 - Embedding additional information in the video signal during the compression process
H04N 19/597 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
38.
METHODS AND SYSTEMS FOR RENDERING OBJECT BASED AUDIO
Methods for generating an object based audio program, renderable in a personalizable manner, and including a bed of speaker channels renderable in the absence of selection of other program content (e.g., to provide a default full range audio experience). Other embodiments include steps of delivering, decoding, and/or rendering such a program. Rendering of content of the bed, or of a selected mix of other content of the program, may provide an immersive experience. The program may include multiple object channels (e.g., object channels indicative of user-selectable and user-configurable objects), the bed of speaker channels, and other speaker channels. Another aspect is an audio processing unit (e.g., encoder or decoder) configured to perform, or which includes a buffer memory which stores at least one frame (or other segment) of an object based audio program (or bitstream thereof) generated in accordance with, any embodiment of the method.
G10L 19/008 - Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
Sampled data is packaged in checkerboard format for encoding and decoding. The sampled data may be quincunx sampled multi-image video data (e.g., 3D video or a multi-program stream), and the data may also be divided into sub-images of each image which are then multiplexed, or interleaved, in frames of a video stream to be encoded and then decoded using a standardized video encoder. A system for viewing may utilize a standard video decoder and a formatting device that de-interleaves the decoded sub-images of each frame reformats the images for a display device. A 3D video may be encoded using a most advantageous interleaving format such that a preferred quality and compression ratio is reached. In one embodiment, the invention includes a display device that accepts data in multiple formats.
H04N 19/597 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
H04N 13/139 - Format conversion, e.g. of frame-rate or size
H04N 13/161 - Encoding, multiplexing or demultiplexing different image signal components
H04N 19/112 - Selection of coding mode or of prediction mode according to a given display mode, e.g. for interlaced or progressive display mode
H04N 19/132 - Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
H04N 19/16 - Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter for a given display mode, e.g. for interlaced or progressive display mode
H04N 19/176 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
H04N 19/33 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability in the spatial domain
H04N 19/46 - Embedding additional information in the video signal during the compression process
H04N 19/587 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
H04N 19/60 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
H04N 19/61 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
H04N 19/85 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
H04N 21/2365 - Multiplexing of several video streams
H04N 21/2383 - Channel coding of digital bit-stream, e.g. modulation
H04N 21/434 - Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams or extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream
H04N 21/438 - Interfacing the downstream path of the transmission network originating from a server, e.g. retrieving MPEG packets from an IP network
A quantization parameter signalling mechanism for both SDR and HDR content in video coding is described using two approaches. The first approach is to send the user-defined QpC table directly in high level syntax. This leads to more flexible and efficient QP control for future codec development and video content coding. The second approach is to signal luma and chroma QPs independently. This approach eliminates the need for QpC tables and removes the dependency of chroma quantization parameter on luma QP.
H04N 19/70 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
H04N 19/172 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
H04N 19/186 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
H04N 19/46 - Embedding additional information in the video signal during the compression process
41.
PERCEPTUALLY-BASED LOSS FUNCTIONS FOR AUDIO ENCODING AND DECODING BASED ON MACHINE LEARNING
Computer-implemented methods for training a neural network, as well as for implementing audio encoders and decoders via trained neural networks, are provided. The neural network may receive an input audio signal, generate an encoded audio signal and decode the encoded audio signal. A loss function generating module may receive the decoded audio signal and a ground truth audio signal, and may generate a loss function value corresponding to the decoded audio signal. Generating the loss function value may involve applying a psychoacoustic model. The neural network may be trained based on the loss function value. The training may involve updating at least one weight of the neural network.
A method of audio processing includes capturing a binaural audio signal, calculating noise reduction gains using a machine learning model, and generating a modified binaural audio signal. The method may further including performing various corrections to the audio to account for video captured by different cameras such as a front camera and a rear camera. The method may further include performing smooth switching of the binaural audio when switching between the front camera and the rear camera. In this manner, noise may be reduced in the binaural audio, and the user perception of the combined video and binaural audio may be improved.
Methods and systems for frame rate scalability are described. Support is provided for input and output video sequences with variable frame rate and variable shutter angle across scenes, or for input video sequences with fixed input frame rate and input shutter angle, but allowing a decoder to generate a video output at a different output frame rate and shutter angle than the corresponding input values. Techniques allowing a decoder to decode more computationally-efficiently a specific backward compatible target frame rate and shutter angle among those allowed are also presented.
H04N 19/31 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability in the temporal domain
H04N 19/172 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
H04N 19/187 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a scalable video layer
H04N 19/30 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
H04N 19/46 - Embedding additional information in the video signal during the compression process
H04N 19/70 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
44.
DETERMINING DIALOG QUALITY METRICS OF A MIXED AUDIO SIGNAL
Disclosed is a method for determining one or more dialog quality metrics of a mixed audio signal comprising a dialog component and a noise component, the method comprising separating an estimated dialog component from the mixed audio signal by means of a dialog separator using a dialog separating model determined by training the dialog separator based on the one or more quality metrics; providing the estimated dialog component from the dialog separator to a quality metrics estimator; and determining the one or more quality metrics by means of the quality metrics estimator based on the mixed signal and the estimated dialog component. Further disclosed is a method for training a dialog separator, a system comprising circuitry configured to perform the method, and a non-transitory computer-readable storage medium.
G10L 25/60 - Speech or voice analysis techniques not restricted to a single one of groups specially adapted for particular use for comparison or discrimination for measuring the quality of voice signals
The present disclose related to a system (1) and method for evaluating the performance of an audio processing scheme. The system (1) comprises an acoustic feature extractor (10A, 10B), configured to receive a plurality of segment pairs, each segment pair comprising a segment (101) and a processed segment (201). The acoustic feature extractor (10A, 10B) determines an acoustic feature associated with each segment and the system (1) further comprises an event detector (11), configured to receive the at least one acoustic feature of each segment (101, 201) and determine, for each segment pair and acoustic feature, if a difference between the acoustic feature of the segment and processed segment exceeds an event threshold. The system also comprises an event analyzer (12), configured to determine a performance metric based on each segment pair associated with a difference exceeding the event threshold.
G10L 25/60 - Speech or voice analysis techniques not restricted to a single one of groups specially adapted for particular use for comparison or discrimination for measuring the quality of voice signals
G10L 25/27 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the analysis technique
Disclosed is a method for separating audio objects in a mixed audio signal, the mixed audio signal comprising a plurality of audio objects. Further disclosed is a computer-implemented method for training a sparse audio object separation model and a method for separating a sparse audio object from a mixed audio signal, the mixed audio signal comprising at least a sparse audio object, a non-sparse audio object, and at least one further audio object. Further disclosed is a computer-implemented method for processing audio based on a signal-to-noise ratio, SNR and a computer-implemented method for processing audio based on a scene environment classification. Disclosed is a non-transitory computer-readable medium and a system configured to perform one or more of the methods.
G10L 21/028 - Voice signal separating using properties of sound source
G10L 19/008 - Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
A method of audio processing includes receiving user-generated content having two audio sources, extracting audio objects and a residual signal, adjusting the audio objects and the residual signal according to the listener's head movements, and mixing the adjusted audio signals to generate a binaural audio signal. In this manner, the binaural signal adjusts according to the listener's head movements without requiring perfect audio objects.
Dual or multi-modulation display systems comprising a first modulator and a second modulator are disclosed. The first modulator may comprise a plurality of analog mirrors (e.g. MEMS array) and the second modulator may comprise a plurality of mirrors (e.g., DMD array). The display system may further comprise a controller that sends control signals to the first and second modulator. The display system may render highlight features within a projected image by affecting a time multiplexing scheme. In one embodiment, the first modulator may be switched on a sub-frame basis such that a desired proportion of the available light may be focused or directed onto the second modulator to form the highlight feature on a sub-frame rendering basis.
A video encoding method according to an embodiment of the present invention includes generating header information that includes information about resolutions of motion vectors of respective blocks, determined based on motion prediction for a unit image. Here, the header information includes flag information indicating whether resolutions of all motion vectors included in the unit image are integer-pixel resolutions. Further, a video decoding method according to another embodiment of the present invention includes extracting information about resolutions of motion vectors of each unit image from header information included in a target bitstream to be decoded; and a decoding unit for decoding the unit image based on the resolution information. Here, the header information includes flag information indicating whether resolutions of all motion vectors included in the unit image are integer-pixel resolutions.
H04N 19/105 - Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
H04N 19/136 - Incoming video signal characteristics or properties
H04N 19/17 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
H04N 19/27 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding involving both synthetic and natural picture components, e.g. synthetic natural hybrid coding [SNHC]
H04N 19/50 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
H04N 19/51 - Motion estimation or motion compensation
H04N 19/523 - Motion estimation or motion compensation with sub-pixel accuracy
H04N 19/70 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
50.
SMART DIALOGUE ENHANCEMENT BASED ON NON-ACOUSTIC MOBILE SENSOR INFORMATION
Described herein is a method of performing environment-aware processing of audio data for a mobile device. In particular, the method may comprise obtaining non-acoustic sensor information of the mobile device. The method may further comprise determining scene information indicative of an environment of the mobile device based on the non-acoustic sensor information. The method may yet further comprise performing audio processing of the audio data based on the determined scene information.
Disclosed herein are techniques for processing streaming data. In some embodiments, the techniques involve obtaining input data representative of a frame of streaming data. The techniques may involve identifying a query transformation, a key transformation, and a value transformation based on the input data. The techniques may involve updating a query buffer, a key buffer, and a value buffer, such that the buffers are each configured to store parameters associated with previous frames of streaming data and the frame of streaming data. The techniques may involve retrieving one or more query frames from the query buffer. The techniques may involve determining a dot product of the query frames and frames in the key buffer to determine a set of weights. The techniques may involve determining a weighted sum between the set of weights and frames in the value buffer, and utilizing the weighted sum to generate a streaming attention vector.
Some disclosed methods involve: receiving an observation sequence including a plurality of extracted features, each of which corresponds to a sequential signal of a sequence of sequential signals; determining a lattice of posterior possibilities, the lattice including a possibility of each observation sequence corresponding to one label class of a plurality of label classes; and applying a loss function to the lattice of posterior possibilities according to ground truth values, where applying the loss function involves applying both sequential information and cluster boundary information. Some methods involve updating parameters for determining the lattice according to losses determined by the loss function and performing the foregoing operations until one or more convergence criteria are met.
In a method to improve backwards compatibility when decoding high-dynamic range images coded in a wide color gamut (WCG) space which may not be compatible with legacy color spaces, hue and/or saturation values of images in an image database are computed for both a legacy color space (say, YCbCr-gamma) and a preferred WCG color space (say, IPT-PQ). Based on a cost function, a reshaped color space is computed so that the distance between the hue values in the legacy color space and rotated hue values in the preferred color space is minimized HDR images are coded in the reshaped color space. Legacy devices can still decode standard dynamic range images assuming they are coded in the legacy color space, while updated devices can use color reshaping information to decode HDR images in the preferred color space at full dynamic range.
H04N 19/87 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving scene cut or scene change detection in combination with video compression
H04N 19/46 - Embedding additional information in the video signal during the compression process
H04N 19/85 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
Some methods may involve receiving a first content stream that includes first audio signals, rendering the first audio signals to produce first audio playback signals, generating first direct sequence spread spectrum (DSSS) signals, generating first modified audio playback signals by inserting the first DSSS signals into the first audio playback signals, and causing a loudspeaker system to play back the first modified audio playback signals, to generate first audio device playback sound. The method(s) may involve receiving microphone signals corresponding to at least the first audio device playback sound and to second through Nth audio device playback sound corresponding to second through Nth modified audio playback signals (including second through Nth DSSS signals) played back by second through Nth audio devices, extracting second through Nth DSSS signals from the microphone signals and estimating at least one acoustic scene metric based, at least partly, on the second through Nth DSSS signals.
Improved methods and/or apparatus for decoding an encoded audio signal in soundfield format for L loudspeakers. The method and/or apparatus can render an Ambisonics format audio signal to 2D loudspeaker setup(s) based on a rendering matrix. The rendering matrix has elements based on loudspeaker positions and wherein the rendering matrix is determined based on weighting at least an element of a first matrix with a weighting factor
Improved methods and/or apparatus for decoding an encoded audio signal in soundfield format for L loudspeakers. The method and/or apparatus can render an Ambisonics format audio signal to 2D loudspeaker setup(s) based on a rendering matrix. The rendering matrix has elements based on loudspeaker positions and wherein the rendering matrix is determined based on weighting at least an element of a first matrix with a weighting factor
ℊ
=
1
L
.
Improved methods and/or apparatus for decoding an encoded audio signal in soundfield format for L loudspeakers. The method and/or apparatus can render an Ambisonics format audio signal to 2D loudspeaker setup(s) based on a rendering matrix. The rendering matrix has elements based on loudspeaker positions and wherein the rendering matrix is determined based on weighting at least an element of a first matrix with a weighting factor
ℊ
=
1
L
.
The first matrix is determined based on positions of the L loudspeakers and at least a virtual position of at least a virtual loudspeaker that is added to the positions of the L loudspeakers.
H04S 3/02 - Systems employing more than two channels, e.g. quadraphonic of the matrix type, i.e. in which input signals are combined algebraically, e.g. after having been phase shifted with respect to each other
H04S 7/00 - Indicating arrangements; Control arrangements, e.g. balance control
A method of audio processing includes performing spatial analysis on a binaural signal to estimate level differences and phase differences characteristic of a binaural filter of the binaural signal, performing object extraction on the binaural audio signal using the estimated level and phase differences to generate a left/right main component signal and a left/right residual component signal. The system may process the left/right main and left/right residual components differently using different object processing parameters for e.g. repositioning, equalization, compression, upmixing, channel remapping or storage to generate a processed binaural signal that provides an improved listening experience. Repositioning may be based on head tracking sensor data.
A method for delivering media content to one or more clients over a distributed system is disclosed. The method may include generating a plurality of network-coded symbols from a plurality of original symbols representing a first media asset. The method may further include generating an original plurality of coded variants of the first media asset. The method may further include distributing a first coded variant of the original plurality of coded variants to a first cache on a first server device for storage in the first cache. The method may further include distributing a second coded variant of the original plurality of coded variants to a second cache on a second server device for storage in the second cache.
H04N 21/60 - Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client; Communication details between server and client
Methods are described to communicate source color volume information in a coded bitstream using SEI messaging. Such data include at least the minimum, maximum, and average luminance values in the source data plus optional data that may include the color volume x and y chromaticity coordinates for the input color primaries (e.g., red, green, and blue) of the source data, and the color x and y chromaticity coordinates for the color primaries corresponding to the minimum, average, and maximum luminance values in the source data. Messaging data signaling an active region in each picture may also be included.
H04N 19/70 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
H04N 19/14 - Coding unit complexity, e.g. amount of activity or edge presence estimation
H04N 19/186 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
H04N 19/20 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding
59.
METHOD FOR ENCODING AND DECODING IMAGE USING ADAPTIVE DEBLOCKING FILTERING, AND APPARATUS THEREFOR
Disclosed is an encoding/decoding method and apparatus related to adaptive deblocking filtering. There is provided an image decoding method performing adaptive filtering in inter-prediction, the method including: reconstructing, from a bitstream, an image signal including a reference block on which block matching is performed in inter-prediction of a current block to be encoded; obtaining, from the bitstream, a flag indicating whether the reference block exists within a current picture where the current block is positioned; reconstructing the current block by using the reference block; adaptively applying an in-loop filter for the reconstructed current block based on the obtained flag; and storing the current block to which the in-loop filter is or is not applied in a decoded picture buffer (DPB).
H04N 19/82 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals - Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation involving filtering within a prediction loop
H04N 19/60 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
H04N 19/51 - Motion estimation or motion compensation
H04N 19/117 - Filters, e.g. for pre-processing or post-processing
H04N 19/50 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
H04N 19/58 - Motion compensation with long-term prediction, i.e. the reference frame for a current frame not being the temporally closest one
H04N 19/176 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
H04N 19/172 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
H04N 19/577 - Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
H04N 19/70 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
H04N 19/137 - Motion inside a coding unit, e.g. average field, frame or block difference
H04N 19/593 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial prediction techniques
H04N 19/107 - Selection of coding mode or of prediction mode between spatial and temporal predictive coding, e.g. picture refresh
H04N 19/184 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being bits, e.g. of the compressed video stream
H04N 19/91 - Entropy coding, e.g. variable length coding [VLC] or arithmetic coding
Methods, systems, and media for enhancing audio content are provided. In some embodiments, a method for enhancing audio content involves receiving a multi-channel audio signal from a first audio capture device and a binaural audio signal from a second audio capture device. The method may further involve extracting one or more objects from the multi-channel audio signal. The method may further involve generating a spatial enhancement mask based on spatial information associated with the one or more objects. The method may further involve applying the spatial enhancement mask to the binaural audio signal to enhance spatial characteristics of the binaural audio signal to generate an enhanced binaural audio signal. The method may further involve generating output binaural audio signal based on the enhanced binaural audio signal.
An apparatus and method of generating personalized HRTFs. The system is prepared by calculating a model for HRTFs described as the relationship between a finite example set of input data, namely anthropometric measures and demographic information for a set of individuals, and a corresponding set of output data, namely HRTFs numerically simulated using a high-resolution database of 3D scans of the same set of individuals. At the time of use, the system queries the user for their demographic information, and then from a series of images of the user, the system detects and measures various anthropometric characteristics. The system then applies the prepared model to the anthropometric and demographic data as part of generating a personalized HRTF. In this manner, the personalized HRTF can be generated with more convenience than by performing a high-resolution scan or an acoustic measurement of the user, and with less computational complexity than by numerically simulating their HRTF.
Some methods may involve receiving a first content stream that includes first audio signals, rendering the first audio signals to produce first audio playback signals, generating first direct sequence spread spectrum (DSSS) signals, generating first modified audio playback signals by inserting the first DSSS signals into the first audio playback signals, and causing a loudspeaker system to play back the first modified audio playback signals, to generate first audio device playback sound. The method(s) may involve receiving microphone signals corresponding to at least the first audio device playback sound and to second through Nth audio device playback sound corresponding to second through Nth modified audio playback signals (including second through Nth DSSS signals) played back by second through Nth audio devices, extracting second through Nth DSSS signals from the microphone signals and estimating at least one acoustic scene metric based, at least partly, on the second through Nth DSSS signals.
Techniques and corresponding systems for estimating an audiogram for a user of a media playback device including obtaining user hearing threshold data for the user, sample hearing threshold data, at least one of sample calibration data and sample noise data, and determining an estimate of the audiogram for the user based on such data. Related techniques for estimating calibration data for a media playback device, as well as corresponding computing apparatus, computer programs, and computer-readable storage media are also described.
A system is programmed to build a machine learning model that comprises a series of masking blocks. Each masking block receives a certain feature vector of an audio segment. Each masking block comprises a first component that generates a first mask for extracting clean speech and a second component that generates a second mask for extracting residual speech masked by the first mask. Each masking block also generates a specific feature vector based on the first mask and the second mask, which becomes the certain feature vector for the next masking block. The second component, which may comprise a gated recurrent unit layer, is computationally less complex than the first component, which may comprise multiple convolutional layers. Furthermore, the system is programmed to receive an input feature vector of an input audio segment and execute the machine learning model to obtain an output feature vector of an output audio segment.
A method of audio content identification includes using a two-stage classifier. The first stage includes previously-existing classifiers and the second stage includes a new classifier. The outputs of the first and second stages calculated over different time periods are combined to generate a steering signal. The final classification results from a combination of the steering signal and the outputs of the first and second stages. In this manner, a new classifier may be added without disrupting existing classifiers.
Disclosed is a method for managing acoustic feedback in real-time audio communications in a communications system, the method comprising determining, by means of a detection module, whether a first communication device is in loudspeaker mode, whether the first communication device is in real-time audio communications with a second communication, and whether the first communication device and the second communication device are in a same acoustic space. Upon determining that this is the case a request signal for requesting one or more measures against acoustic feedback is provided to a mitigation module. Further disclosed are a device and a system configured to perform the method, a non-transitory computer-readable medium, an encoder and a decoder.
H04M 9/08 - Two-way loud-speaking telephone systems with means for conditioning the signal, e.g. for suppressing echoes for one or both directions of traffic
Volume leveler controller and controlling method are disclosed. In one embodiment, A volume leveler controller includes an audio content classifier for identifying the content type of an audio signal in real time; and an adjusting unit for adjusting a volume leveler in a continuous manner based on the content type as identified. The adjusting unit may configured to positively correlate the dynamic gain of the volume leveler with informative content types of the audio signal, and negatively correlate the dynamic gain of the volume leveler with interfering content types of the audio signal.
G10L 25/30 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the analysis technique using neural networks
G10L 25/51 - Speech or voice analysis techniques not restricted to a single one of groups specially adapted for particular use for comparison or discrimination
G10L 21/0364 - Speech enhancement, e.g. noise reduction or echo cancellation by changing the amplitude for improving intelligibility
68.
METHOD AND DEVICE FOR DECODING A HIGHER-ORDER AMBISONICS (HOA) REPRESENTATION OF AN AUDIO SOUNDFIELD
The invention discloses rendering sound field signals, such as Higher-Order Ambisonics (HOA), for arbitrary loudspeaker setups, where the rendering results in highly improved localization properties and is energy preserving. This is obtained by rendering an audio sound field representation for arbitrary spatial loudspeaker setups and/or by a a decoder that decodes based on a decode matrix (D). The decode matrix (D) is based on smoothing and scaling of a first decode matrix {circumflex over (D)} with smoothing coefficients. The first decode matrix {circumflex over (D)} is based on a mix matrix G and a mode matrix {tilde over (ψ)}, where the mix matrix G was determined based on L speakers and positions of a spherical modelling grid related to a HOA order N, and the mode matrix {tilde over (ψ)} was determined based on the spherical modelling grid and the HOA order N.
Encoding/decoding an audio signal having one or more audio components, wherein each audio component is associated with a spatial location. A first audio signal presentation (z) of the audio components, a first set of transform parameters (w(f)), and signal level data (β2) are encoded and transmitted to the decoder. The decoder uses the first set of transform parameters (w(f)) to form a reconstructed simulation input signal intended for an acoustic environment simulation, and applies a signal level modification (α) to the reconstructed simulation input signal. The signal level modification is based on the signal level data (β2) and data (p2) related to the acoustic environment simulation. The attenuated reconstructed simulation input signal is then processed in an acoustic environment simulator. With this process, the decoder does not need to determine the signal level of the simulation input signal, thereby reducing processing load.
G10L 19/008 - Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
G10L 19/00 - Speech or audio signal analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
G10L 19/02 - Speech or audio signal analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
70.
SPATIAL AUDIO RENDERING ADAPTIVE TO SIGNAL LEVEL AND LOUDSPEAKER PLAYBACK LIMIT THRESHOLDS
Rendering audio signals may involve a mapping for each audio signal to the loudspeaker signals computed as a function of an audio signal's intended perceived spatial position, physical positions associated with the loudspeakers and a time- and frequency-varying representation of loudspeaker signal level relative to a maximum playback limit of each loudspeaker. Each mapping may be computed to approximately achieve the intended perceived spatial position of an associated audio signal when the loudspeaker signals are played back. A representation of loudspeaker signal level relative to a maximum playback limit may be computed for each audio signal. The mapping of an audio signal into a particular loudspeaker signal may be reduced as loudspeaker signal level relative to a maximum playback limit increases above a threshold, while the mapping may be increased into one or more other loudspeakers for which the maximum playback limits are less than a threshold.
The positions of a plurality of speakers at a media consumption site are determined. Audio information in an object-based format is received. Gain adjustment value for a sound content portion in the object-based format may be determined based on the position of the sound content portion and the positions of the plurality of speakers. Audio information in a ring-based channel format is received. Gain adjustment value for each ring-based channel in a set of ring-based channels may be determined based on the ring to which the ring-based channel belongs and the positions of the speakers at a media consumption site.
Methods and systems for frame rate scalability are described. Support is provided for input and output video sequences with variable frame rate and variable shutter angle across scenes, or for input video sequences with fixed input frame rate and input shutter angle, but allowing a decoder to generate a video output at a different output frame rate and shutter angle than the corresponding input values. Techniques allowing a decoder to decode more computationally-efficiently a specific backward compatible target frame rate and shutter angle among those allowed are also presented.
H04N 19/31 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability in the temporal domain
H04N 19/187 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a scalable video layer
H04N 19/172 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
H04N 19/30 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
H04N 19/46 - Embedding additional information in the video signal during the compression process
H04N 19/70 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
73.
MULTIPLE-INTENT COMPOSITE IMAGE ENCODING AND RENDERING
Techniques for multiple-intent composite image encoding and rendering are disclosed. The techniques can include obtaining a set of constituent images for a composite image, determining a common rendering intent to be applied to the set of constituent images, adjusting one or more of the set of constituent images according to the common rendering intent, resulting in an adjusted set of constituent images, creating the composite image based on the adjusted set of constituent images, generating metadata characterizing the common rendering intent, and encoding the composite image and the metadata to create an encoded multiple-intent composite image.
H04N 19/46 - Embedding additional information in the video signal during the compression process
H04N 19/597 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
H04N 19/85 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
G06T 11/60 - Editing figures and text; Combining figures or text
74.
PROCESSING OF EXTENDED DIMENSION LIGHT FIELD IMAGES
In one embodiment, methods, media, and systems process and display light field images using a view function that is based on pixel locations in the image and on the viewer's distance (observer's Z position) from the display. The view function can be an angular view function that specifies different angular views for different pixels in the light field image based on the inputs that can include: the x or y pixel location in the image, the viewer's distance from the display, and the viewer's angle relative to the display. In one embodiment, light field metadata, such as angular range metadata and/or angular offset metadata can be used to process and display the image. In one embodiment, color volume mapping metadata can be used to adjust color volume mapping based on the determined angular views; and the color volume mapping metadata can also be adjusted based on angular offset metadata.
H04N 13/117 - Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
H04N 13/366 - Image reproducers using viewer tracking
H04N 13/388 - Volumetric displays, i.e. systems where the image is built up from picture elements distributed through a volume
H04N 13/232 - Image signal generators using stereoscopic image cameras using a single 2D image sensor using fly-eye lenses, e.g. arrangements of circular lenses
H04N 13/178 - Metadata, e.g. disparity information
H04N 23/957 - Light-field or plenoptic cameras or camera modules
75.
COLOR-GRADING CONTENT BASED ON SIMILARITY TO EXEMPLARS
Systems and methods for color grading of images and video based on similarity to exemplars. In preparation for color-grading new content, exemplar frames related to the expected new content may be obtained and color grading parameters for the exemplar frames may be obtained. To color grade the new content as it is created or received, similarities between frames of the new content and the exemplar frames may be determined. The similarities between frames of the new content and the exemplar frames may be determined may be combined with the obtained color-grading parameters from the exemplar frames to determine suitable color-grading parameters to apply to the new content. The new content may then be color-graded using the determined color-grading parameters.
H04N 9/64 - Circuits for processing colour signals
H04N 9/77 - Circuits for processing the brightness signal and the chrominance signal relative to each other, e.g. adjusting the phase of the brightness signal relative to the colour signal, correcting differential gain or differential phase
76.
Alias cancelling during audio coding mode transitions
An apparatus for processing an audio signal and method thereof are disclosed. The present invention includes receiving, by an audio processing apparatus, an audio signal including a first data of a first block encoded with rectangular coding scheme and a second data of a second block encoded with non-rectangular coding scheme; receiving a compensation signal corresponding to the second block; estimating a prediction of an aliasing part using the first data; and, obtaining a reconstructed signal for the second block based on the second data, the compensation signal and the prediction of aliasing part.
G10L 19/00 - Speech or audio signal analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
G10L 25/45 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the type of analysis window
G10L 21/00 - Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
G10L 19/04 - Speech or audio signal analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
G10L 19/022 - Blocking, i.e. grouping of samples in time; Choice of analysis windows; Overlap factoring
In an embodiment, a method comprises: filtering reference audio content items to separate the reference audio content items into different frequency bands; for each frequency band, extracting a first feature vector from at least a portion of each of the reference audio content items, wherein the first feature vector includes at least one audio characteristic of the reference audio content items; obtaining at least one semantic label from at least a portion of each of the reference audio content items; obtaining a second feature vector consisting of the first feature vectors per frequency band and the at least one semantic label; generating, based on the second feature vector, cluster feature vectors representing centroids of clusters; separating the reference audio content items according to the cluster feature vectors; and computing an average target profile for each cluster based on the reference audio content items in the cluster.
H03G 5/00 - Tone control or bandwidth control in amplifiers
H04R 3/04 - Circuits for transducers for correcting frequency response
G10L 25/21 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the type of extracted parameters the extracted parameters being power information
G10L 25/18 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the type of extracted parameters the extracted parameters being spectral information of each sub-band
G10L 15/18 - Speech classification or search using natural language modelling
78.
SYSTEM FOR MAINTAINING REVERSIBLE DYNAMIC RANGE CONTROL INFORMATION ASSOCIATED WITH PARAMETRIC AUDIO CODERS
On the basis of a bitstream (P), an n-channel audio signal (X) is reconstructed by deriving an m-channel core signal (Y) and multichannel coding parameters (a) from the bitstream, where 1≤m
E21B 33/138 - Plastering the borehole wall; Injecting into the formation
E21B 41/00 - Equipment or details not covered by groups
E21B 21/00 - Methods or apparatus for flushing boreholes, e.g. by use of exhaust air from motor
G10L 19/008 - Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
G10L 19/24 - Variable rate codecs, e.g. for generating different qualities using a scalable representation such as hierarchical encoding or layered encoding
A method may involve: receiving direction of arrival (DOA) data corresponding to sound emitted by at least a first smart audio device of the audio environment that includes a first audio transmitter and a first audio receiver, the DOA data corresponding to sound received by at least a second smart audio device of the audio environment that includes a second audio transmitter and a second audio receiver, the DOA data corresponding to sound emitted by at least the second smart audio device and received by at least the first smart audio device; receiving one or more configuration parameters corresponding to the audio environment, to one or more audio devices, or both; and minimizing a cost function based at least in part on the DOA data and the configuration parameter(s), to estimate a position and an orientation of at least the first smart audio device and the second smart audio device.
Disclosed are methods and systems which convert a multi-microphone input signal to a multichannel output signal making use of a time- and frequency-varying matrix. For each time and frequency tile, the matrix is derived as a function of a dominant direction of arrival and a steering strength parameter. Likewise, the dominant direction and steering strength parameter are derived from characteristics of the multi-microphone signals, where those characteristics include values representative of the inter-channel amplitude and group-delay differences.
H04R 1/40 - Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only by combining a number of identical transducers
Methods, systems, bitstream syntax, and fixed-point implementations are described for loop filtering using neural networks in image and video processing. Given an input image, a hybrid luma-chroma filter is proposed, wherein luma and chroma components are first processed by a first neural network and the output of the first network is subsequently processed by separate luma and chroma subnetworks. Finally, the outputs of the separate luma and chroma subnetworks are concatenated to generate the filtered output of the input image. Computational efficient methods using CP-decomposition are also described. Methods indicating the position of the neural-net loop filter relatively to other filters, such as the adaptive loop filter (ALF) are also discussed.
H04N 19/82 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals - Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation involving filtering within a prediction loop
In some embodiments, a method, comprises: dividing, using at least one processor, an audio input into speech and non-speech segments; for each frame in each non-speech segment, estimating, using the at least one processor, a time-varying noise spectrum of the non-speech segment; for each frame in each speech segment, estimating, using the at least one processor, speech spectrum of the speech segment; for each frame in each speech segment, identifying one or more non-speech frequency components in the speech spectrum; comparing the one or more non-speech frequency components with one or more corresponding frequency components in a plurality of estimated noise spectra and selecting the estimated noise spectrum from the plurality of estimated noise spectra based on a result of the comparing.
G10L 21/028 - Voice signal separating using properties of sound source
G10L 25/18 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the type of extracted parameters the extracted parameters being spectral information of each sub-band
G10L 25/84 - Detection of presence or absence of voice signals for discriminating voice from noise
G10L 21/0364 - Speech enhancement, e.g. noise reduction or echo cancellation by changing the amplitude for improving intelligibility
G10L 25/21 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the type of extracted parameters the extracted parameters being power information
83.
ROTATION OF SOUND COMPONENTS FOR ORIENTATION-DEPENDENT CODING SCHEMES
Method for encoding scene-based audio is provided. In some implementations, the method involves determining, by an encoder, a spatial direction of a dominant sound component in a frame of an input audio signal. In some implementations, the method involves determining rotation parameters based on the determined spatial direction and a direction preference of a coding scheme to be used to encode the input audio signal. In some implementations, the method involves rotating sound components of the frame based on the rotation parameters such that, after being rotated, the dominant sound component has a spatial direction that aligns with the direction preference of the coding scheme. In some implementations, the method involves encoding the rotated sound components of the frame of the input audio signal using the coding scheme in connection with an indication of the rotation parameters or an indication of the spatial direction of the dominant sound component.
G10L 19/008 - Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
Methods and systems for frame rate scalability are described. Support is provided for input and output video sequences with variable frame rate and variable shutter angle across scenes, or for input video sequences with fixed input frame rate and input shutter angle, but allowing a decoder to generate a video output at a different output frame rate and shutter angle than the corresponding input values. Techniques allowing a decoder to decode more computationally-efficiently a specific backward compatible target frame rate and shutter angle among those allowed are also presented.
H04N 19/31 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability in the temporal domain
H04N 19/187 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a scalable video layer
H04N 19/172 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
H04N 19/30 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
H04N 19/46 - Embedding additional information in the video signal during the compression process
H04N 19/70 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
85.
METHODS AND DEVICES FOR ENCODING AND/OR DECODING IMMERSIVE AUDIO SIGNALS
The present document describes a method (700) for encoding a multi-channel input signal (201). The method (700) comprises determining (701) a plurality of downmix channel signals (203) from the multi-channel input signal (201) and performing (702) energy compaction of the plurality of downmix channel signals (203) to provide a plurality of compacted channel signals (404). Furthermore, the method (700) comprises determining (703) joint coding metadata (205) based on the plurality of compacted channel signals (404) and based on the multi-channel input signal (201), wherein the joint coding metadata (205) is such that it allows upmixing of the plurality of compacted channel signals (404) to an approximation of the multi-channel input signal (201). In addition, the method (700) comprises encoding (704) the plurality of compacted channel signals (404) and the joint coding metadata (205).
G10L 19/008 - Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
An input image of a first bit depth in an input domain is received. Forward reshaping operations are performed on the input image to generate a forward reshaped image of a second bit depth in a reshaping domain. An image container containing image data derived from the forward reshaped image is encoded into an output video signal of the second bit depth.
H04N 19/105 - Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
H04N 19/186 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
87.
INTER CODING USING DEEP LEARNING IN VIDEO COMPRESSION
Methods, systems, and bitstream syntax are described for inter-frame coding using end-to-end neural networks used in image and video compression. Inter-frame coding methods include one or more of: joint luma-chroma motion compensation for YUV pictures, joint luma-chroma residual coding for YUV pictures, using attention layers, enabling temporal motion prediction networks for motion vector prediction, using a cross-domain network which combines motion vector and residue information for motion vectors decoding, using the cross-domain network for decoding residuals, using weighted motion-compensated inter prediction, and using temporal only, spatial only, or both temporal and spatial features in entropy decoding. Methods to improve training of neural networks for inter-frame coding are also described.
H04N 19/70 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
H04N 19/90 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups , e.g. fractals
The present disclosure relates to a method and system for processing audio, as well as a computer program product comprising instructions which, when the program is executed by a computer, causes the computer to carry out the method. The method comprises obtaining an input audio signal and processing the input audio signal to extract a height audio object from the input audio signal, wherein the height audio object is extracted using a source separation module configured to extract an audio object of a predetermined height audio source type. The method further comprises rendering the input audio signal to a multi-channel presentation such that the at least one height audio object is included in at least one height channel of the multi-channel presentation.
H04S 5/00 - Pseudo-stereo systems, e.g. in which additional channel signals are derived from monophonic signals by means of phase shifting, time delay or reverberation
elsdd. With the proposed approach, a "soft" separation of the real AIR into an early AIR and a late AIR. Specifically, the early AIR will decay to zero during a transition period d, while the late AIR will gradually increase from zero during the transition period. The sum of the early AIR and late AIR will still be equal to the real AIR.
H04S 7/00 - Indicating arrangements; Control arrangements, e.g. balance control
G10L 25/30 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the analysis technique using neural networks
90.
FRAME-LEVEL PERMUTATION INVARIANT TRAINING FOR SOURCE SEPARATION
Described is a method of training a deep-learning-based system for sound source separation. The system comprises a separation stage for frame-wise extraction of representations of sound sources from a representation of an audio signal, and a clustering stage for generating, for each frame, a vector indicative of an assignment permutation of extracted frames of representations of sound sources to respective sound sources. The representation of the audio signal is a waveform-based representation. The separation stage is trained using frame-level permutation invariant training. Further, the clustering stage is trained to generate embedding vectors for the frames of the audio signal that allow to determine estimates of respective assignment permutations between extracted sound signals and labels of sound sources that had been used for the frames. Also described is a method of using the deep-learning-based system for sound source separation.
A method for compressing a HOA signal being an input HOA representation with input time frames (C(k)) of HOA coefficient sequences comprises spatial HOA encoding of the input time frames and subsequent perceptual encoding and source encoding. Each input time frame is decomposed (802) into a frame of predominant sound signals (XPS(k−1)) and a frame of an ambient HOA component ({tilde over (C)}AMB(k−1)). The ambient HOA component ({tilde over (C)}AMB(k−1)) comprises, in a layered mode, first HOA coefficient sequences of the input HOA representation (cn(k−1)) in lower positions and second HOA coefficient sequences (cAMB,n(k−1)) in remaining higher positions. The second HOA coefficient sequences are part of an HOA representation of a residual between the input HOA representation and the HOA representation of the predominant sound signals.
H04S 3/00 - Systems employing more than two channels, e.g. quadraphonic
G10L 19/008 - Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
G10L 19/24 - Variable rate codecs, e.g. for generating different qualities using a scalable representation such as hierarchical encoding or layered encoding
H04S 7/00 - Indicating arrangements; Control arrangements, e.g. balance control
92.
SIGNAL RESHAPING AND CODING FOR HDR AND WIDE COLOR GAMUT SIGNALS
In a method to improve the coding efficiency of high-dynamic range (HDR) images, a decoder parses sequence processing set (SPS) data from an input coded bitstream to detect that an HDR extension syntax structure is present in the parsed SPS data. It extracts from the HDR extension syntax structure post-processing information that includes one or more of a color space enabled flag, a color enhancement enabled flag, an adaptive_reshaping_enabled_flag, a dynamic range conversion flag, a color correction enabled flag, or an SDR_viewable_flag. It decodes the input bitstream to generate a preliminary output decoded signal, and generates a second output signal based on the preliminary output signal and the post-processing information.
H04N 19/70 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
H04N 19/186 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
H04N 19/30 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
H04N 19/85 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
Image-processing technique directed at improving the quality of viewable images generated by rendering a multiplane image having a plurality of pixels and represented by a plurality of layers corresponding to different respective distances from the reference camera position. In an example embodiment, the image-processing technique includes one or more of the following operations: (A) for a first set of pixels, scaling respective weights of the layers to cause a sum of the scaled weights to be normalized to one; (B) for a second set of pixels, replacing respective alpha and texture values in the layers by the corresponding local average values; and (C) for a third set of pixels, scaling corresponding texture values in the layers such that, for the resulting viewable image rendered for the reference camera position, texture values of the third set match the respective texture values of the source image captured from the reference camera position.
Embodiments are disclosed for timestamp smoothing to remove jitter. In some embodiments, a method of smoothing timestamps associated with audio packets comprises: receiving, using at least one processor, a series of input timestamps for audio packets and their respective packet lengths; estimating, using the at least one processor, an initial timestamp based on the series of input timestamps, the packet lengths and a sample time; calculating, using the at least one processor, a predicted timestamp based on the estimated initial timestamp; and smoothing, using the at least one processor, the predicted timestamp.
H04L 47/283 - Flow control; Congestion control in relation to timing considerations in response to processing delays, e.g. caused by jitter or round trip time [RTT]
H04L 43/106 - Active monitoring, e.g. heartbeat, ping or trace-route using time related information in packets, e.g. by adding timestamps
H04L 41/147 - Network analysis or design for predicting network behaviour
95.
SUBBAND DOMAIN ACOUSTIC ECHO CANCELLER BASED ACOUSTIC STATE ESTIMATOR
Some implementations involve receiving, from a first subband domain acoustic echo canceller (AEC) of a first audio device in an audio environment, first adaptive filter management data from each of a plurality of first adaptive filter management modules, each first adaptive filter management module corresponding to a subband of the first subband domain AEC, each first adaptive filter management module being configured to control a first plurality of adaptive filters. The first plurality of adaptive filters may include at least a first adaptive filter type and a second adaptive filter type. Some implementations involve extracting, from the first adaptive filter management data, a first plurality of extracted features corresponding to a plurality of subbands of the first subband domain AEC and estimating a current local acoustic state based, at least in part, on the first plurality of extracted features.
A method, an apparatus, and logic to post-process raw gains determined by input processing to generate post-processed gains, comprising using one or both of delta gain smoothing and decision-directed gain smoothing. The delta gain smoothing comprises applying a smoothing filter to the raw gain with a smoothing factor that depends on the gain delta: the absolute value of the difference between the raw gain for the current frame and the post-processed gain for a previous frame. The decision-directed gain smoothing comprises converting the raw gain to a signal-to-noise ratio, applying a smoothing filter with a smoothing factor to the signal-to-noise ratio to calculate a smoothed signal-to-noise ratio, and converting the smoothed signal-to-noise ratio to determine the second smoothed gain, with smoothing factor possibly dependent on the gain delta.
An encoding device evaluates a plurality of processing and/or post-processing algorithms and/or methods to be applied to a video stream, and signals a selected method, algorithm, class or category of methods/algorithms either in an encoded bitstream or as side information related to the encoded bitstream. A decoding device or post-processor utilizes the signaled algorithm or selects an algorithm/method based on the signaled method or algorithm. The selection is based, for example, on availability of the algorithm/method at the decoder/post-processor and/or cost of implementation. The video stream may comprise, for example, downsampled multiplexed stereoscopic images and the selected algorithm may include any of upconversion and/or error correction techniques that contribute to a restoration of the downsampled images.
H04N 19/597 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
H04N 19/80 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals - Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation
H04N 19/17 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
H04N 13/161 - Encoding, multiplexing or demultiplexing different image signal components
H04N 13/172 - Processing image signals image signals comprising non-image signal components, e.g. headers or format information
H04N 13/178 - Metadata, e.g. disparity information
H04N 13/218 - Image signal generators using stereoscopic image cameras using a single 2D image sensor using spatial multiplexing
H04N 19/154 - Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion
H04N 19/85 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
H04N 19/44 - Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
H04N 19/895 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving methods or arrangements for detection of transmission errors at the decoder in combination with error concealment
H04N 19/70 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
H04N 19/86 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving reduction of coding artifacts, e.g. of blockiness
H04N 21/434 - Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams or extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream
H04N 19/176 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
H04N 19/587 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
H04N 19/172 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
H04N 19/423 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals - characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation characterised by memory arrangements
An encoding device evaluates a plurality of processing and/or post-processing algorithms and/or methods to be applied to a video stream, and signals a selected method, algorithm, class or category of methods/algorithms either in an encoded bitstream or as side information related to the encoded bitstream. A decoding device or post-processor utilizes the signaled algorithm or selects an algorithm/method based on the signaled method or algorithm. The selection is based, for example, on availability of the algorithm/method at the decoder/post-processor and/or cost of implementation. The video stream may comprise, for example, downsampled multiplexed stereoscopic images and the selected algorithm may include any of upconversion and/or error correction techniques that contribute to a restoration of the downsampled images.
H04N 19/597 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
H04N 19/80 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals - Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation
H04N 19/17 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
H04N 13/161 - Encoding, multiplexing or demultiplexing different image signal components
H04N 13/172 - Processing image signals image signals comprising non-image signal components, e.g. headers or format information
H04N 13/178 - Metadata, e.g. disparity information
H04N 13/218 - Image signal generators using stereoscopic image cameras using a single 2D image sensor using spatial multiplexing
H04N 19/154 - Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion
H04N 19/85 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
H04N 19/44 - Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
H04N 19/895 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving methods or arrangements for detection of transmission errors at the decoder in combination with error concealment
H04N 19/70 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
H04N 19/86 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving reduction of coding artifacts, e.g. of blockiness
H04N 21/434 - Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams or extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream
H04N 19/176 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
H04N 19/587 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
H04N 19/172 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
H04N 19/423 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals - characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation characterised by memory arrangements
99.
MULTI-HALF-TONE IMAGING AND DUAL MODULATION PROJECTION/DUAL MODULATION LASER PROJECTION
Smaller halftone tiles are implemented on a first modulator of a dual modulation projection system. This techniques uses multiple halftones per frame in the pre-modulator synchronized with a modified bit sequence in the primary modulator to effectively increase the number of levels provided by a given tile size in the halftone modulator. It addresses the issue of reduced contrast ratio at low light levels for small tile sizes and allows the use of smaller PSFs which reduce halo artifacts in the projected image and may be utilized in 3D projecting and viewing.
H04N 9/31 - Projection devices for colour picture display
G09G 3/20 - Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix
Methods, systems, and media for processing audio are provided. In some embodiments, a method involves receiving, from a plurality of microphones, an input audio signal. The method may involve identifying an angle of arrival associated with the input audio signal. The method may involve determining a plurality of gains corresponding to a plurality of bands of the input audio signal based on a combination of at least: 1) a representation of a covariance of signals associated with microphones of the plurality of microphones on a per-band basis; and 2) the angle of arrival. The method may involve applying the plurality of gains to the plurality of bands of the input audio signal such that at least a portion of the input audio signal is suppressed to form an enhanced audio signal.
G10L 21/0264 - Noise filtering characterised by the type of parameter measurement, e.g. correlation techniques, zero crossing techniques or predictive techniques
G10L 21/0216 - Noise filtering characterised by the method used for estimating noise
