09 - Scientific and electric apparatus and instruments
35 - Advertising and business services
42 - Scientific, technological and industrial services, research and design
Goods & Services
Downloadable computer programs and downloadable computer
software using artificial intelligence for natural language
processing, generation, understanding and analysis;
downloadable computer programs and downloadable computer
software for machine learning; downloadable computer
programs and downloadable computer software for image
recognition and generation; downloadable computer programs
and downloadable computer software using artificial
intelligence for music generation and suggestions;
downloadable computer programs and downloadable computer
software for artificial intelligence, namely, computer
software for developing, running and analyzing algorithms
that are able to learn to analyze, classify, and take
actions in response to exposure to data; downloadable
computer software using artificial intelligence for image
and video editing and retouching; downloadable computer
software using artificial intelligence for the generation of
text, images, photos, videos, audio, and multimedia content;
downloadable computer software using artificial intelligence
for connecting consumers with targeted promotional
advertisements; downloadable computer software using
artificial intelligence for the generation of advertisements
and promotional materials; downloadable computer software
using artificial intelligence for creating and generating
text; downloadable computer software using artificial
intelligence for translating words or text from one language
to another; downloadable chatbot software for image
recognition and generation; downloadable chatbot software
for music generation; downloadable chatbot software for
image and video editing and retouching; downloadable chatbot
software for the generation of text, images, photos, videos,
audio and multimedia content; downloadable chatbot software
for connecting consumers with promotional messaging;
downloadable chatbot software for simulating human
conversations; downloadable chatbot software for suggesting
image, video, audio, text, and multimedia content;
downloadable chatbot software for responding to oral and
written prompts. Advertising, marketing, and promotion services; marketing,
advertising, and promotional services using artificial
intelligence software, chatbot software, and augmented
reality software; dissemination of advertising for others
via computer and other communication networks; online retail
store services featuring a wide variety of consumer goods of
others; promoting the goods and services of others by
providing an internet website portal featuring links to the
websites of others; facilitating the exchange and sale of
services and products of third parties via computer and
communication networks, namely, operating on-line
marketplaces for sellers and buyers of goods and services;
consumer profiling for commercial or marketing purposes;
providing consumer information and advice for consumers in
the selection of products to buy. Research and development in the field of artificial
intelligence; providing online non-downloadable software
using artificial intelligence for natural language
processing, generation, understanding, and analysis;
providing online non-downloadable software for developing,
running and analyzing algorithms that are able to learn to
analyze, classify, and take actions in response to exposure
to data; software as a service (saas) services featuring
software for using language models; providing online
non-downloadable software for machine-learning based
language and speech processing; providing online
non-downloadable software for the translation text from one
language to another; providing on-line non-downloadable
software using artificial intelligence for image recognition
and generation; providing on-line non-downloadable software
using artificial intelligence for text recognition and
generation; providing online non-downloadable software for
the generation of advertisements and promotional materials;
providing on-line non-downloadable software using artificial
intelligence for music generation and suggestions; providing
on-line non-downloadable software using artificial
intelligence for image and video editing and retouching;
providing on-line non-downloadable software using artificial
intelligence for the generation of text, images, photos,
videos, audio, and multimedia content; providing on-line
non-downloadable software using artificial intelligence for
connecting consumers with promotional advertisements;
providing temporary use of online non-downloadable chatbot
software using artificial intelligence for image recognition
and generation; providing temporary use of online
non-downloadable chatbot software using artificial
intelligence for text recognition and generation; providing
temporary use of online non-downloadable chatbot software
using artificial intelligence for music recognition and
generation; providing temporary use of online
non-downloadable chatbot software using artificial
intelligence for the generation of text, images, photos,
video, audio, text, and multimedia content; providing
temporary use of online non-downloadable chatbot software
using artificial intelligence for connecting consumers with
advertisements; providing temporary use of online
non-downloadable chatbot software using artificial
intelligence for simulating human conversations; providing
temporary use of online non-downloadable chatbot software
using artificial intelligence for responding to oral and
written prompts.
A case for a portable device like a smartphone includes light sources such as LEDs, which, when illuminated, can be detected and tracked by a head-worn augmented or virtual reality device. The light sources may be located at the corners of the case and may emit infrared light. A relative pose between the smartphone and the head-worn device can be determined based on computer vision techniques performed on images captured by the head-worn device that includes light from the light sources. Relative movement between the smartphone and the head-worn device can be used to provide user input to the head-worn device, as can touch input on the portable device. In some instances, the case is powered inductively from the portable device.
H04M 1/72409 - User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality by interfacing with external accessories
H04B 1/3888 - Arrangements for carrying or protecting transceivers
H04M 1/72454 - User interfaces specially adapted for cordless or mobile telephones with means for adapting the functionality of the device according to specific conditions according to context-related or environment-related conditions
A method for carving a 3D space using hands tracking is described. In one aspect, a method includes accessing a first frame from a camera of a display device, tracking, using a hand tracking algorithm operating at the display device, hand pixels corresponding to one or more user hands depicted in the first frame, detecting, using a sensor of the display device, depths of the hand pixels, identifying a 3D region based on the depths of the hand pixels, and applying a 3D reconstruction engine to the 3D region.
A case for a portable device like a smartphone includes light sources such as LEDs, which, when illuminated, can be detected and tracked by a head-worn augmented or virtual reality device. The light sources may be located at the corners of the case and may emit infrared light. A relative pose between the smartphone and the head-worn device can be determined based on computer vision techniques performed on images captured by the head- worn device that includes light from the light sources. Relative movement between the smartphone and the head-worn device can be used to provide user input to the head-worn device, as can touch input on the portable device. In some instances, the case is powered inductively from the portable device.
G06F 3/00 - Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
5.
MULTIFUNCTIONAL CASE FOR ELECTRONICS-ENABLED EYEWEAR
A carry case for an electronics-enabled eyewear device has incorporated therein electronic components for connection to the eyewear device while storing the eyewear device. The case comprises a rigid frame structure defining an openable holding space for the pair of smart glasses, and a compressible shock-resistant protective cover on the frame structure. The exterior of the case may be predominantly defined by the shock resistant protective cover.
Systems, methods, and computer readable media for a service manger to manage services on a wearable device are disclosed. The service manager remains active in memory and listens for requests for services. The service manager then determines which services to run and which to stop to respond to the requests for services. After running a service, the service manager calls the service to respond to the request and sends a response to the request to the sender of the request. The service manager may be resident on a different processor than a processor from which the requests for services originate. The service manager maintains priorities of the services to determine which services to stop or remove from memory.
Systems and methods of generating ground truth datasets for producing virtual reality (VR) experiences, for testing simulated sensor configurations, and for training machine-learning algorithms. In one example, a recording device with one or more cameras and one or more inertial measurement units captures images and motion data along a real path through a physical environment. A SLAM application uses the captured data to calculate the trajectory of the recording device. A polynomial interpolation module uses Chebyshev polynomials to generate a continuous time trajectory (CTT) function. The method includes identifying a virtual environment and assembling a simulated sensor configuration, such as a VR headset. Using the CTT function, the method includes generating a ground truth output dataset that represents the simulated sensor configuration in motion along a virtual path through the virtual environment. The virtual path is closely correlated with the motion along the real path as captured by the recording device. Accordingly, the output dataset produces a realistic and life-like VR experience. In addition, the methods described can be used to generate multiple output datasets, at various sample rates, which are useful for training the machine-learning algorithms which are part of many VR systems.
A display-enabled eyewear device has an integrated head sensor that dynamically and continuously measures or detects various cephalic parameters of a wearer's head. The head sensor includes a loop coupler system integrated in a lens-carrying frame to sense proximate ambient RF absorption influenced by head presence, size, and/or distance. Autonomous device management dynamically adjust or cause adjustment of selected device features based on current detected values for the cephalic parameters, which can include wear status, head size, and frame-head spacing.
Methods and systems are disclosed for performing real-time stylizing operations. The system receives an image that includes a depiction of a whole body of a real-world person. The system applies a machine learning model to the image to generate a stylized version of the whole body of the real-world person corresponding to a given style, the machine learning model being trained using training data to establish a relationship between a plurality of training images depicting synthetically rendered whole bodies of persons and corresponding ground-truth stylized versions of the whole bodies of the persons of the given style. The system replaces the depiction of the whole body of the real-world person in the image with the generated stylized version of the whole body of the real-world person.
A method for recognizing sign language using collaborative augmented reality devices is described. In one aspect, a method includes accessing a first image generated by a first augmented reality device and a second image generated by a second augmented reality device, the first image and the second image depicting a hand gesture of a user of the first augmented reality device, synchronizing the first augmented reality device with the second augmented reality device, in response to the synchronizing, distributing one or more processes of a sign language recognition system between the first and second augmented reality devices, collecting results from the one or more processes from the first and second augmented reality devices, and displaying, in near real-time in a first display of the first augmented reality device, text indicating a sign language translation of the hand gesture based on the results.
Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for providing reduced availability modes in messaging. The program and method provide for maintaining a count of consecutive time periods in which message content has been exchanged between a first user and a second user in a messaging application; receiving, from a device associated with the first user, a request to set an availability mode for the first user to a reduced availability mode with respect to the messaging application; setting, in response to receiving the request, the availability mode for the first user to the reduced availability mode; and refraining from updating the count while the availability mode is set to the reduced availability mode.
H04L 51/043 - Real-time or near real-time messaging, e.g. instant messaging [IM] using or handling presence information
H04L 51/224 - Monitoring or handling of messages providing notification on incoming messages, e.g. pushed notifications of received messages
H04L 51/52 - User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail for supporting social networking services
12.
HEAD PROPERTY DETECTION IN DISPLAY-ENABLED WEARABLE DEVICES
A display-enabled eyewear device has an integrated head sensor that dynamically and continuously measures or detects various cephalic parameters of a wearer's head. The head sensor includes a loop coupler system integrated in a lens-carrying frame to sense proximate ambient RF absorption influenced by head presence, size, and/or distance. Autonomous device management dynamically adjust or cause adjustment of selected device features based on current detected values for the cephalic parameters, which can include wear status, head size, and frame-head spacing.
A method for recognizing sign language using collaborative augmented reality devices is described. In one aspect, a method includes accessing a first image generated by a first augmented reality device and a second image generated by a second augmented reality device, the first image and the second image depicting a hand gesture of a user of the first augmented reality device, synchronizing the first augmented reality device with the second augmented reality device, in response to the synchronizing, distributing one or more processes of a sign language recognition system between the first and second augmented reality devices, collecting results from the one or more processes from the first and second augmented reality devices, and displaying, in near real-time in a first display of the first augmented reality device, text indicating a sign language translation of the hand gesture based on the results.
G06F 40/58 - Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G06F 3/0346 - Pointing devices displaced or positioned by the user; Accessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
G06V 10/26 - Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
G06V 20/20 - Scenes; Scene-specific elements in augmented reality scenes
G06V 40/10 - Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
G06V 40/20 - Movements or behaviour, e.g. gesture recognition
An energy-efficient adaptive 3D sensing system. The adaptive 3D sensing system includes one or more cameras and one or more projectors. The adaptive 3D sensing system captures images of a real-world scene using the one or more cameras and computes depth estimates and depth estimate confidence values for pixels of the images. The adaptive 3D sensing system computes an attention mask based on the one or more depth estimate confidence values and commands the one or more projectors to send a distributed laser beam into one or more areas of the real-world scene based on the attention mask. The adaptive 3D sensing system captures 3D sensing image data of the one or more areas of the real-world scene and generates 3D sensing data for the real-world scene based on the 3D sensing image data.
Systems, devices, media, and methods are presented for determining a level of abusive network behavior suspicion for groups of entities and for identifying suspicious entity groups. A suspiciousness metric is developed and used to evaluate a multi-view graph across multiple views where entities are associated with nodes of the graph and attributes of the entities are associated with levels of the graph.
A computer-implement method comprises: training a classifier with labeled data from a dataset; classifying, by the trained classifier, unlabeled data from the dataset; providing, by the classifier to a policy gradient, a reward signal for each data/query pair; transferring, by the classifier to a ranker, learning; training, by the policy gradient, the ranker; ranking data from the dataset based on a query; and retrieving data from the ranked data in response to the query.
A method for enhancing a presentation of a network document by a client terminal with real time social media content. The method comprises analyzing a content in a web document to identify a relation to a first of a plurality of multi participant events documented in an event dataset, each of the plurality of multi participant events is held in a geographical venue which hosts an audience of a plurality of participants, matching a plurality of event indicating tags of each of a plurality of user uploaded media content files with at least one feature of the first multi participant event to identify a group of user uploaded media content files selected from the plurality of user uploaded media content files, and forwarding at least some members of the group to a simultaneous presentation on a browser running on a client terminal and presenting the web document.
A hand-tracking platform generates gesture components for use as user inputs into an application of an Augmented Reality (AR) system. In some examples, the hand-tracking platform generates real-world scene environment frame data based on gestures being made by a user of the AR system using a camera component of the AR system. The hand-tracking platform recognizes a gesture component based on the real-world scene environment frame data and generates gesture component data based on the gesture component. The application utilizes the gesture component data as user input in a user interface of the application.
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G06F 3/00 - Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
G06T 19/00 - Manipulating 3D models or images for computer graphics
An energy-efficient adaptive 3D sensing system. The adaptive 3D sensing system includes one or more cameras and one or more projectors. The adaptive 3D sensing system captures images of a real-world scene using the one or more cameras and computes depth estimates and depth estimate confidence values for pixels of the images. The adaptive 3D sensing system computes an attention mask based on the one or more depth estimate confidence values and commands the one or more projectors to send a distributed laser beam into one or more areas of the real-world scene based on the attention mask. The adaptive 3D sensing system captures 3D sensing image data of the one or more areas of the real-world scene and generates 3D sensing data for the real-world scene based on the 3D sensing image data.
H04N 5/222 - Studio circuitry; Studio devices; Studio equipment
G01B 11/25 - Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. moiré fringes, on the object
H04N 23/56 - Cameras or camera modules comprising electronic image sensors; Control thereof provided with illuminating means
G06F 3/00 - Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
A display driver device (210) receives a downloadable “sequence” for dynamically reconfiguring displayed image characteristics in an image system. The display driver device comprises one or more storage devices, for example, memory devices, for storing image data (218) and portions of drive sequences (219) that are downloadable and/or updated in real time depending on various inputs (214).
G09G 3/20 - Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix
21.
HYPEREXTENDING HINGE FOR WEARABLE ELECTRONIC DEVICE
Eyewear having a frame, a hinge, and a hyperextendable temple. An extender is coupled to the hinge and the temple, and the extender extends with respect to the hinge allowing hyperextension of the temple with respect to the frame. The extender may include a bushing and a spring that allows the temple hyperextension, and which also creates a bias force to urge the temple against a user's head during use.
Methods and systems are disclosed for performing real-time stylizing operations. The system receives an image that includes a depiction of a whole body of a real-world person. The system applies a machine learning model to the image to generate a stylized version of the whole body of the real-world person corresponding to a given style, the machine learning model being trained using training data to establish a relationship between a plurality of training images depicting synthetically rendered whole bodies of persons and corresponding ground-truth stylized versions of the whole bodies of the persons of the given style. The system replaces the depiction of the whole body of the real-world person in the image with the generated stylized version of the whole body of the real-world person.
Systems, methods, and computer readable media for voice-controlled user interfaces (UIs) for augmented reality (AR) wearable devices are disclosed. Embodiments are disclosed that enable a user to interact with the AR wearable device without using physical user interface devices. An application has a non-voice-controlled UI mode and a voice-controlled UI mode. The user selects the mode of the UI. The application running on the AR wearable device displays UI elements on a display of the AR wearable device. The UI elements have types. Predetermined actions are associated with each of the UI element types. The predetermined actions are displayed with other information and used by the user to invoke the corresponding UI element.
G06F 3/04817 - Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance using icons
G06F 3/0484 - Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
G10L 15/16 - Speech classification or search using artificial neural networks
G10L 15/22 - Procedures used during a speech recognition process, e.g. man-machine dialog
24.
REMOTE ANNOTATION AND NAVIGATION USING AN AR WEARABLE DEVICE
Systems, methods, and computer readable media for remote annotations, drawings, and navigation instructions sent to an augmented reality (AR) wearable device from a computing device are disclosed. The AR wearable device captures images and sends them to the remote computing device to provide a real-time view of what the user of the AR wearable device sees. A user of the remote computing device can add navigation instructions and can select an image to annotate or draw on. The AR wearable device provides 3-dimensional (3D) coordinate information within a 3D world of the AR wearable device for the selected image. The user of the remote computing device then annotates or draws on the selected image. The remote computing device determines 3D coordinates for the annotations and drawings within the 3D world of the AR wearable device. The annotations and drawings are sent to the AR wearable device with associated 3D coordinates.
Systems, methods, and computer readable media for remote annotations, drawings, and navigation instructions sent to an augmented reality (AR) wearable device from a computing device are disclosed. The AR wearable device captures images and sends them to the remote computing device to provide a real-time view of what the user of the AR wearable device sees. A user of the remote computing device can add navigation instructions and can select an image to annotate or draw on. The AR wearable device provides 3-dimensional (3D) coordinate information within a 3D world of the AR wearable device for the selected image. The user of the remote computing device then annotates or draws on the selected image. The remote computing device determines 3D coordinates for the annotations and drawings within the 3D world of the AR wearable device. The annotations and drawings are sent to the AR wearable device with associated 3D coordinates.
A UAV having a wireless-front end including propellers that are dual purposed to function as ground communication antenna elements. This design reduces weight and size of the UAV, hence enabling a compact design with the capability of handling a heavier payload.
Aspects of the present disclosure involve a system for presenting AR items. The system receives a video that includes a depiction of a real-world object in a real-world environment. The system generates a three-dimensional (3D) bounding box for the real-world object and stabilizes the 3D bounding box based on one or more sensors of the device. The system determines a position, orientation, and dimensions of the real-world object based on the stabilized 3D bounding box and renders a display of an augmented reality (AR) item within the video based on the position, orientation, and dimensions of the real-world object.
Methods and systems are disclosed for performing generating AR experiences on a messaging platform. The methods and systems receive, from a client device, a request to access an augmented reality (AR) experience and access a list of event types associated with the AR experience used to generate one or more metrics. The methods and systems determine that an interaction associated with the AR experience corresponds to a first event type of the list of event types and generates interaction data for the first event type representing the interaction. In response to receiving a request to terminate the AR experience, the systems and methods transmit the interaction data to a remote server.
A system for deformation or bending correction in an Augmented Reality (AR) system. Sensors are positioned in a frame of a head-worn AR system to sense forces or pressure acting on the frame by temple pieces attached to the frame. The sensed forces or pressure are used in conjunction with a model of the frame to determine a corrected model of the frame. The corrected model is used to correct video data captured by the AR system and to correct a video virtual overlay that is provided to a user wearing the head- worn AR system.
Systems and methods are provided for performing operations on an augmented reality (AR) device using an external screen streaming system. The system establishes, by one or more processors of an AR device, a communication with an external client device. The system causes overlay of, by the AR device, a first AR object on a real-world environment being viewed using the AR device. The system receives, by the AR device, a first image from the external client device. The system, in response to receiving the first image from the external client device, overlays the first image on the first AR object by the AR device.
A system including a drone having a projector to project an image from a projection origin. The drone also has a navigation unit to determine location information for the drone. A processor coupled to the drone includes a memory. Execution of programming by the processor configures the system to obtain a projection surface architecture for a projection surface. The projection surface architecture includes reference points that correspond to physical locations on the projection surface. Each reference point is associated with relationship data with respect to an architecture origin. The system also receives location information for the drone, adapts the relationship data responsive to change in the location information, adjusts the image using the adapted relationship data, and projects the adjusted image onto the projection surface.
Embodiments described herein include an expressive icon system to present an animated graphical icon, wherein the animated graphical icon is generated by capture facial tracking data at a client device. In some embodiments, the system may track and capture facial tracking data of a user via a camera associated with a client device (e.g., a front facing camera, or a paired camera), and process the facial tracking data to animate a graphical icon.
G06T 13/40 - 3D [Three Dimensional] animation of characters, e.g. humans, animals or virtual beings
G06F 3/04817 - Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance using icons
G06V 40/16 - Human faces, e.g. facial parts, sketches or expressions
H04M 1/72427 - User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality for supporting games or graphical animations
H04M 1/7243 - User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality with interactive means for internal management of messages
H04M 1/72469 - User interfaces specially adapted for cordless or mobile telephones for operating the device by selecting functions from two or more displayed items, e.g. menus or icons
Methods and systems for videoconferencing include generating work quality metrics based on emotion recognition of an individual such as a call center agent. The work quality metrics allow for workforce optimization. One example method includes the steps of receiving a video including a sequence of images, detecting an individual in one or more of the images, locating feature reference points of the individual, aligning a virtual face mesh to the individual in one or more of the images based at least in part on the feature reference points, dynamically determining over the sequence of images at least one deformation of the virtual face mesh, determining that the at least one deformation refers to at least one facial emotion selected from a plurality of reference facial emotions, and generating quality metrics including at least one work quality parameter associated with the individual based on the at least one facial emotion.
G06Q 10/0639 - Performance analysis of employees; Performance analysis of enterprise or organisation operations
G06V 40/16 - Human faces, e.g. facial parts, sketches or expressions
H04N 21/4402 - Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
Aspects of the present disclosure involve a system for performing real-time in-painting using machine learning techniques. The system receives a video that includes a depiction of a real-world object in a real -world environment. The system accesses a segmentation associated with the real-world object and removes a depiction of the real -world object from a region of a first frame of the video. The system processes, by a machine learning model, the first frame and one or more previous frames of the video that precede the first frame to generate a new frame in which portions of the first frame have been blended into the region from which the depiction of the real-world object has been removed.
Systems and methods are provided for performing operations on an augmented reality (AR) device using an external screen streaming system. The system establishes, by one or more processors of an AR device, a communication with an external client device. The system causes overlay of, by the AR device, a first AR object on a real -world environment being viewed using the AR device. The system receives, by the AR device, a first image from the external client device. The system, in response to receiving the first image from the external client device, overlays the first image on the first AR object by the AR device.
An augmented reality (AR) content system is provided. The AR content system may analyze audio input obtained from a user to generate a search request. The AR content system may obtain search results in response to the search request and determine a layout by which to display the search results. The search results may be displayed in a user interface within an AR environment according to the layout. The AR content system may also analyze audio input to detect commands to perform with respect to content displayed in the user interface.
An augmented reality (AR) content system is provided. The AR content system may analyze audio input obtained from a user to generate a search request. The AR content system may obtain search results in response to the search request and determine a layout by which to display the search results. The search results may be displayed in a user interface within an AR environment according to the layout. The AR content system may also analyze audio input to detect commands to perform with respect to content displayed in the user interface.
A method of providing an interactive personal mobility system, performed by one or more processors, comprises determining an initial pose by visual-inertial odometry performed on images and inertial measurement unit (IMU) data generated by a wearable augmented reality device. Sensor data transmitted from a personal mobility system is received, and sensor fusion is performed on the data received from the personal mobility system to provide an updated pose. Augmented reality effects are displayed on the wearable augmented reality device based on the updated pose.
B60L 15/20 - Methods, circuits or devices for controlling the propulsion of electrically-propelled vehicles, e.g. their traction-motor speed, to achieve a desired performance; Adaptation of control equipment on electrically-propelled vehicles for remote actuation from a stationary place, from alternative parts of the vehicle or from alternative vehicles of the same vehicle train for control of the vehicle or its driving motor to achieve a desired performance, e.g. speed, torque, programmed variation of speed
G01C 22/02 - Measuring distance traversed on the ground by vehicles, persons, animals or other moving solid bodies, e.g. using odometers or using pedometers by conversion into electric waveforms and subsequent integration, e.g. using tachometer generator
A system and method are described for generating 3D garments from two-dimensional (2D) scribble images drawn by users. The system includes a conditional 2D generator, a conditional 3D generator, and two intermediate media including dimension-coupling color-density pairs and flat point clouds that bridge the gap between dimensions. Given a scribble image, the 2D generator synthesizes dimension-coupling color-density pairs including the RGB projection and density map from the front and rear views of the scribble image. A density-aware sampling algorithm converts the 2D dimension-coupling color-density pairs into a 3D flat point cloud representation, where the depth information is ignored. The 3D generator predicts the depth information from the flat point cloud. Dynamic variations per garment due to deformations resulting from a wearer's pose as well as irregular wrinkles and folds may be bypassed by taking advantage of 2D generative models to bridge the dimension gap in a non-parametric way.
A mixed-reality media content system may be configured to perform operations that include: causing display of image data at a client device, the image data comprising a depiction of an object that includes a graphical code at a position upon the object; detecting the graphical code at the position upon the depiction of the object based on the image data; accessing media content within a media repository based on the graphical code scanned by the client device; and causing display of a presentation of the media content at the position of the graphical code upon the depiction of the object at the client device.
G06T 19/00 - Manipulating 3D models or images for computer graphics
G06T 19/20 - Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
G06K 19/06 - Record carriers for use with machines and with at least a part designed to carry digital markings characterised by the kind of the digital marking, e.g. shape, nature, code
G06F 3/0481 - Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
Aspects of the present disclosure involve a system for performing real-time in-painting using machine learning techniques. The system receives a video that includes a depiction of a real-world object in a real-world environment. The system accesses a segmentation associated with the real-world object and removes a depiction of the real-world object from a region of a first frame of the video. The system processes, by a machine learning model, the first frame and one or more previous frames of the video that precede the first frame to generate a new frame in which portions of the first frame have been blended into the region from which the depiction of the real-world object has been removed.
A system for deformation or bending correction in an Augmented Reality (AR) system. Sensors are positioned in a frame of a head-worn AR system to sense forces or pressure acting on the frame by temple pieces attached to the frame. The sensed forces or pressure are used in conjunction with a model of the frame to determine a corrected model of the frame. The corrected model is used to correct video data captured by the AR system and to correct a video virtual overlay that is provided to a user wearing the head-worn AR system.
A method for prohibiting email content propagation that receives, at a server, an email message. At the server, at least one email address associated with the email message which is designated not to receive a content of the email message is identified. At the server, the email message is modified by selectively removing a content of the email message to be conveyed to the at least one email address. The server conveys the modified email message to the at least one email address. The server conveys the email message to one or more recipient email addresses except the at least one email address. Consequently, the server has sent a submitted message to multiple email addresses, while modifying the content sent to a subset of the addresses that received the email message.
H04L 51/212 - Monitoring or handling of messages using filtering or selective blocking
G06F 3/04847 - Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
G06Q 10/107 - Computer-aided management of electronic mailing [e-mailing]
G06Q 50/00 - Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
H04L 51/214 - Monitoring or handling of messages using selective forwarding
H04L 51/48 - Message addressing, e.g. address format or anonymous messages, aliases
H04L 51/52 - User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail for supporting social networking services
Aspects of the present disclosure involve a system for presenting AR items. The system receives a video that includes a depiction of a real-world object in a real-world environment. The system generates a three-dimensional (3D) bounding box for the real-world object and stabilizes the 3D bounding box based on one or more sensors of the device. The system determines a position, orientation, and dimensions of the real-world object based on the stabilized 3D bounding box and renders a display of an augmented reality (AR) item within the video based on the position, orientation, and dimensions of the real -world object.
A UAV having a manual gimbal including a camera, and a flight mode selector configured to select both a flight mode and manually establish a camera position as a function of the selected fight mode. A controller responds to a position of the gimbal or selector to establish the flight mode. The flight mode is selected from several available modes, for example, a horizontal flight mode, a 45-degree flight mode, and a vertical (aerial) flight mode. The flight mode selector is mechanically coupled to the gimbal and establishes a pitch angle of the gimbal, and thus the camera angle attached to the gimbal.
B64C 39/02 - Aircraft not otherwise provided for characterised by special use
B64C 19/00 - Aircraft control not otherwise provided for
B64U 101/30 - UAVs specially adapted for particular uses or applications for imaging, photography or videography
G05D 1/00 - Control of position, course, altitude, or attitude of land, water, air, or space vehicles, e.g. automatic pilot
G05D 1/08 - Control of attitude, i.e. control of roll, pitch, or yaw
G06F 3/0362 - Pointing devices displaced or positioned by the user; Accessories therefor with detection of 1D translations or rotations of an operating part of the device, e.g. scroll wheels, sliders, knobs, rollers or belts
B64U 50/14 - Propulsion using external fans or propellers ducted or shrouded
A mixed-reality media content system may be configured to perform operations that include: causing display of image data at a client device, the image data comprising a depiction of an object that includes a graphical code at a position upon the object; detecting the graphical code at the position upon the depiction of the object based on the image data; accessing media content within a media repository based on the graphical code scanned by the client device; and causing display of a presentation of the media content at the position of the graphical code upon the depiction of the object at the client device.
H04N 21/8545 - Content authoring for generating interactive applications
G06F 3/04817 - Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance using icons
G06K 19/06 - Record carriers for use with machines and with at least a part designed to carry digital markings characterised by the kind of the digital marking, e.g. shape, nature, code
Systems and methods herein describe privacy preserving multi-touch attribution. The described systems access a plurality of impression events and a plurality of conversion events, and for each impression event and each conversion event, wherein each impression event and each conversion event are associated with user identifiers, the described systems generates a hashed user identifier based on the associated user identifier, initiates a key agreement protocol comprising a key, generates an encrypted identifier by encrypting the hashed user identifier with the key, and stores the encrypted identifier.
H04L 9/30 - Public key, i.e. encryption algorithm being computationally infeasible to invert and users' encryption keys not requiring secrecy
H04L 9/32 - Arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system
H04L 29/06 - Communication control; Communication processing characterised by a protocol
48.
DYNAMICALLY ASSIGNING PARTICIPANT VIDEO FEEDS WITHIN VIRTUAL CONFERENCING SYSTEM
Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for dynamically assigning participant video feeds within a virtual conferencing system. The program and method provide, in association with designing a virtual space for virtual conferencing, an interface for configuring a set of rooms, each room being associated with a different number of participant video elements assignable to respective participant video feeds; receive, via the interface, an indication of user input for setting properties for the set of rooms; determine, in association with virtual conferencing, a first number of participants for a room; select a first room corresponding to the first number of participants; provide display of the first room; and assign, for each of the first number of participants, a participant video feed corresponding to the participant with a respective participant video element in the first room.
G06F 3/04815 - Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
G06F 3/0482 - Interaction with lists of selectable items, e.g. menus
G06F 3/0484 - Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
An augmented reality (AR) eyewear device has a lens system which includes an optical screening mechanism that enables switching the lens system between a conventional see-through state and an opaque state in which the lens system screens or functionally blocks out the wearer's view of the external environment. Such a screening mechanism allows for expanded use cases of the AR glasses compared to conventional devices, e.g.: as a sleep mask; to view displayed content like movies or sports events against a visually non-distracting background instead of against the external environment; and/or to enable VR functionality.
A system for hand tracking for an Augmented Reality (AR) system. The AR system uses a camera of the AR system to capture tracking video frame data of a hand of a user of the AR system. The AR system generates a skeletal model based on the tracking video frame data and determines a location of the hand of the user based on the skeletal model. The AR system causes a steerable camera of the AR system to focus on the hand of the user.
Embodiments described herein relate to an augmented expression system to generate and cause display of a specially configured interface to present an augmented reality perspective. The augmented expression system receives image and video data of a user and tracks facial landmarks of the user based on the image and video data, in real-time to generate and present a 3-dimensional (3D) bitmoji of the user.
Among other things, embodiments of the present disclosure improve the functionality of electronic messaging systems by enabling users in an electronic chat conversation to run applications together. In some embodiments, when one user in a chat launches an application, an icon or other visual representation of the application appears in a portion of the chat window (e.g., in a “chat dock”) for other users in the chat to access.
H04L 65/401 - Support for services or applications wherein the services involve a main real-time session and one or more additional parallel real-time or time sensitive sessions, e.g. white board sharing or spawning of a subconference
Devices, media, and methods are presented for an immersive augmented reality (AR) experience using an eyewear device with spatial audio. The eyewear device has a processor, a memory, an image sensor, and a speaker system. The eyewear device captures image information for an environment surrounding the device and identifies an object location within the same environment. The eyewear device then associates a virtual object with the identified object location. The eyewear device monitors the position of the device with respect to the virtual object and presents audio signals to alert the user that the identified object is in the environment.
The subject technology captures first image data by a computing device, the first image data comprising a target face of a target actor and facial expressions of the target actor, the facial expressions including lip movements. The subject technology generates, based at least in part on frames of a source media content, sets of source pose parameters. The subject technology receives a selection of a particular facial expression from a set of facial expressions. The subject technology generates, based at least in part on sets of source pose parameters and the selection of the particular facial expression, an output media content. The subject technology provides augmented reality content based at least in part on the output media content for display on the computing device.
Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for providing bot participants for virtual conferencing. The program and method provide, in association with designing a virtual space, a first interface for configuring plural participant video elements, each being assignable to a respective participant; receive, via the first interface, an indication of user input for setting first properties for the plural participant video elements; provide a second interface for configuring a bot participant for simulating an actual participant in association with a participant video element of the plural participant video elements; receive, via the second interface, an indication of second user input for setting second properties for the bot participant; and provide, in association with designing the virtual space, display of the virtual space based on the first and second properties, the bot participant being assigned to the participant video element.
G06F 3/04847 - Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
G06F 3/04815 - Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
G06T 5/20 - Image enhancement or restoration by the use of local operators
H04L 51/02 - User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages
H04L 65/403 - Arrangements for multi-party communication, e.g. for conferences
56.
MENU HIERARCHY NAVIGATION ON ELECTRONIC MIRRORING DEVICES
Systems and methods are provided for performing operations comprising: capturing, by an electronic mirroring device, a video feed received from a camera of the electronic mirroring device, the video feed depicting a user; displaying, by one or more processors of the electronic mirroring device, one or more menu options on the video feed that depicts the user, the one or more menu options relating to a first level in a hierarchy of levels; detecting a gesture performed by the user in the video feed; and in response to detecting the gesture, displaying a set of options related to a given option of the one or more menu options, the set of options relating to a second level in the hierarchy of levels.
A drone system is configured to capture an audio stream that includes voice commands from an operator, to process the audio stream for identification of the voice commands, and to perform operations based on the identified voice commands. The drone system can identify a particular voice stream in the audio stream as an operator voice, and perform the command recognition with respect to the operator voice to the exclusion of other voice streams present in the audio stream. The drone can include a directional camera that is automatically and continuously focused on the operator to capture a video stream usable in disambiguation of different voice streams captured by the drone.
AR-enabled wearable electronic devices such as smart glasses are adapted for use as an (Internet of Things) IoT remote control device where the user can control a pointer on a television screen, computer screen, or other IoT enabled device to select items by looking at them and making selections using gestures. Built-in six-degrees-of-freedom (6DoF) tracking capabilities are used to move the pointer on the screen to facilitate navigation. The display screen is tracked in real-world coordinates to determine the point of intersection of the user's view with the screen using raycasting techniques. Hand and head gesture detection are used to allow the user to execute a variety of control actions by performing different gestures. The techniques are particularly useful for smart displays that offer AR-enhanced content that can be viewed in the displays of the AR-enabled wearable electronic devices.
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G06F 3/00 - Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
G06T 19/00 - Manipulating 3D models or images for computer graphics
H04L 67/125 - Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks involving control of end-device applications over a network
A resource optimized kiosk mode that improves the mobile experience for creators and users of mobile devices such as an augmented reality (AR)-enabled wearable eyewear device. An eyewear device enters a kiosk mode by receiving a kiosk mode request for an application and, in response to the request, determining which services and application programming interfaces (APIs) are required to execute the selected application. An identification of the determined services and APIs required to execute the selected application are stored and the eyewear device is rebooted. After reboot, the selected application is started, and only the identified services and APIs are enabled. To determine which services and APIs are required to execute the selected application, metadata may be associated with the selected application specifying the services and/or APIs that the selected application requires to use when in operation.
A gesture-based wake process for an AR system is described herein. The AR system places a hand-tracking input pipeline of the AR system in a suspended mode. A camera component of the hand-tracking input pipeline detects a possible visual wake command being made by a user of the AR system. On the basis of detecting the possible visual wake command, the AR system wakes the hand-tracking input pipeline and places the camera component in a fully operational mode. If the AR system, using the hand¬ tracking input pipeline, verifies the possible visual wake command as an actual wake command, the AR system initiates execution of an AR application.
09 - Scientific and electric apparatus and instruments
42 - Scientific, technological and industrial services, research and design
Goods & Services
Spatial Light Modulators; Displays, namely, liquid crystal displays and liquid crystal-on-silicon displays; Microdisplays, namely, liquid crystal microdisplays and liquid crystal-on-silicon microdisplays; Emissive displays, namely, OLED (Organic light emitting diode) display panels and OLED microdisplays; Micro Light Emitting Diode displays (microLED displays); Liquid Crystal Devices, namely, liquid crystal displays and liquid crystal microdisplays; Liquid Crystal displays; Display Panels, namely, liquid crystal display panels and microdisplay panels, LED display panels and microdisplay panels; Liquid Crystal Modules, namely, liquid crystal displays; Liquid Crystal-on-Silicon (LCoS) devices, namely, liquid crystal-on-silicon (LCOS) panels and micro panels to project digital images and video; Driver Integrated Circuits; computer hardware, namely, microchips, integrated circuits, semiconductor chips, and circuit boards for modulation of electromagnetic radiation (light) and/or image display; driver software, namely, downloadable and/or recorded computer software for allowing communication with a liquid crystal microdisplay, liquid crystal-on-silicon display, emissive display (an OLED or LED display), or Micro Light Emitting Diode display (microLED display); systems software, namely, downloadable and/or recorded software for managing a liquid crystal microdisplay, liquid crystal-on-silicon display, emissive display (an OLED or LED display), or Micro Light Emitting Diode display (microLED display) via the driver; applications software, namely, downloadable or recorded software for playing of application content on a display and for configuring a liquid crystal microdisplay, liquid crystal-on-silicon display, emissive display (an OLED or LED display), or Micro Light Emitting Diode display (microLED display); control software, namely, downloadable, recorded, and/or embedded software to control a liquid crystal microdisplay, liquid crystal-on-silicon display, emissive display (an OLED or LED display), or Micro Light Emitting Diode display (microLED display) and to interpret commands from the driver; configuration software, namely, downloadable and/or recorded software for creating operation configurations for a liquid crystal microdisplay, liquid crystal-on-silicon display, emissive display (an OLED or LED display), or Micro Light Emitting Diode display (microLED display), and for calibrating the operations performed by a liquid crystal microdisplay, liquid crystal-on-silicon display, emissive display (an OLED or LED display), or Micro Light Emitting Diode display (microLED display) Configuration software, namely, providing temporary use of online non-downloadable software for generating configuration parameters (for generating drive sequences) for a liquid crystal microdisplay, liquid crystal-on-silicon display, emissive display (an OLED or LED display), or Micro Light Emitting Diode display (microLED display)
A method of generating an image for use in a conversation taking place in a messaging application is disclosed. Conversation input text is received from a user of a portable device that includes a display. Model input text is generated from the conversation input text, which is processed with a text-to-image model to generate an image based on the model input text. The coordinates of a face in the image are determined, and the face of the user or another person is added to the image at the location. The final image is displayed on the portable device, and user input is received to transmit the image to a remote recipient.
Systems and methods are provided for retrieving first query result data associated with a first user account and rendering the first query result data into a first result item, generating a shareable search result stream comprising the first result item associated with the first user account, retrieving second query result data associated with a second user account and rendering the second query result data into a second result item, adding the second result item to the shareable search result stream associated with the first user account, and providing the sharable search result stream comprising the first result item and the second result item to a first computing device associated with the first user account and a second computing device associated with the second user account.
Systems, methods, and computer readable media for graphical assistance with tasks using an augmented reality (AR) wearable devices are disclosed. Embodiments capture an image of a first user view of a real-world scene and access indications of surfaces and locations of the surfaces detected in the image. The AR wearable device displays indications of the surfaces on a display of the AR wearable device where the locations of the indications are based on the locations of the surfaces and a second user view of the real-world scene. The locations of the surfaces are indicated with 3D world coordinates. The user views are determined based on a location of the user. The AR wearable device enables a user to add graphics to the surfaces and select tasks to perform. Tools such as a bubble level or a measuring tool are available for the user to utilize to perform the task.
A system for hand tracking for an Augmented Reality (AR) system. The AR system uses a camera of the AR system to capture tracking video frame data of a hand of a user of the AR system. The AR system generates a skeletal model based on the tracking video frame data and determines a location of the hand of the user based on the skeletal model. The AR system causes a steerable camera of the AR system to focus on the hand of the user.
A method of generating an image for use in a conversation taking place in a messaging application is disclosed. Conversation input text is received from a user of a portable device that includes a display. Model input text is generated from the conversation input text, which is processed with a text-to-image model to generate an image based on the model input text. The coordinates of a face in the image are determined, and the face of the user or another person is added to the image at the location. The final image is displayed on the portable device, and user input is received to transmit the image to a remote recipient.
An augmented reality (AR) eyewear device has a lens system which includes an optical screening mechanism that enables switching the lens system between a conventional see- through state and an opaque state in which the lens system screens or functionally blocks out the wearer's view of the external environment. Such a screening mechanism allows for expanded use cases of the AR glasses compared to conventional devices, e.g.: as a sleep mask; to view displayed content like movies or sports events against a visually nondistracting background instead of against the external environment; and/or to enable VR functionality.
Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for configuring a three-dimensional (3D) model within a virtual conferencing system. The program and method provide, in association with designing a room for virtual conferencing, an interface for configuring a 3D model; receiving, via the interface, an indication of user input for setting properties for the 3D model, the properties specifying image data for projecting onto the 3D model; and in association with virtual conferencing, providing display of the room based on the properties for the 3D model, and causing the image data to be projected onto the 3D model within the room.
G06F 3/04815 - Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
G06F 3/04845 - Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
G06F 3/04847 - Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
G06T 19/00 - Manipulating 3D models or images for computer graphics
G06T 19/20 - Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
69.
DEVICE AND METHOD FOR COMPENSATING EFFECTS OF PANTOSCOPIC TILT OR WRAP/SWEEP TILT ON AN IMAGE PRESENTED ON AN AUGMENTED REALITY OR VIRTUAL REALITY DISPLAY
An optical device is disclosed for use in an augmented reality or virtual reality display, comprising a waveguide (12; 22; 32) and an input diffractive optical element (H0; H3; 34) positioned in or on the waveguide, configured to receive light from a projector and couple it into the waveguide so that it is captured within the waveguide under total internal reflection. The input diffractive optical element has an input grating vector (G0; Gig) in the plane of the waveguide. The device includes a first diffractive optical element (H1; H4) and a second diffractive optical element (H2; H5) having first and second grating vectors (G2, G3; GV1, GV2) respectively in the plane of the waveguide, wherein the first diffractive optical element is configured to receive light from the input diffractive optical element and to couple it towards the second diffractive optical element, and wherein the second diffractive optical element is configured to receive light from the first diffractive optical element and to couple it out of the waveguide towards a viewer. The input grating vector, the first grating vector and the second grating vector have different respective magnitudes, and wherein a vector addition of the input grating vector, the first grating vector and the second grating vector sums to zero.
A gesture-based wake process for an AR system is described herein. The AR system places a hand-tracking input pipeline of the AR system in a suspended mode. A camera component of the hand-tracking input pipeline detects a possible visual wake command being made by a user of the AR system. On the basis of detecting the possible visual wake command, the AR system wakes the hand-tracking input pipeline and places the camera component in a fully operational mode. If the AR system, using the hand-tracking input pipeline, verifies the possible visual wake command as an actual wake command, the AR system initiates execution of an AR application.
Aspects of the present disclosure involve a system and a method for performing operations comprising: receiving, by a messaging application implemented on a client device, input that selects a sound option to add sound to one or more images; in response to receiving the input, presenting a sound editing user interface element that visually indicates a played portion of the sound and separately visually indicates an un-played portion of the sound; receiving an interaction with the sound editing user interface element to modify a start point of the sound; embedding a graphical element representing the sound in the one or more images; playing, by the messaging application, the sound associated with the graphical element starting from the start point together with displaying the one or more images.
Methods and devices for wired charging and communication with a wearable device are described. In one embodiment, a symmetrical contact interface comprises a first contact pad and a second contact pad, and particular wired circuitry is coupled to the first and second contact pads to enable charging as well as receive and transmit communications via the contact pads as part of various device states.
H01L 27/02 - Devices consisting of a plurality of semiconductor or other solid-state components formed in or on a common substrate including integrated passive circuit elements with at least one potential-jump barrier or surface barrier
H01R 13/62 - Means for facilitating engagement or disengagement of coupling parts or for holding them in engagement
H02J 7/04 - Regulation of the charging current or voltage
H02J 7/34 - Parallel operation in networks using both storage and other dc sources, e.g. providing buffering
H03K 19/0185 - Coupling arrangements; Interface arrangements using field-effect transistors only
H04B 3/56 - Circuits for coupling, blocking, or by-passing of signals
Systems and methods herein describe a method for capturing a video in real-time by an image capture device. The system provides a plurality of visual pose hints, identifies first pose information in the video while capturing the video, applies a first series of virtual effects to the video, identifies second pose information, and applies a second series of virtual effects to the video, the second series of virtual effects based on the first series of virtual effects.
A method and a system include receiving a request from a client device to view a media content item, determining at least one comment associated with a respective user profile from a set of connected profiles, generating a summary comments selectable item based at least in part on the respective user profile, causing a display of playback of the media content item and the summary comments selectable item in response to the request to view the media content item, and during the playback of the media content item at the particular time, causing a display of at least one comment.
H04L 51/52 - User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail for supporting social networking services
G06F 3/04817 - Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance using icons
A system includes a communication module that receives a request to post content to an event gallery associated with an event. The request in turn includes geo-location data for a device sending the content, and identification data identifying the device or a user of the device. The system further has an event gallery module to perform a first authorization operation that includes determining that the geo-location data corresponds to a geo-location fence associated with an event. The event gallery module also performs a second authorization operation that includes using the identification data to verify an attribute of the user. Finally, based on the first and second authorization operations, the event gallery module may selectively authorize the device to post the content to the event gallery.
H04L 51/222 - Monitoring or handling of messages using geographical location information, e.g. messages transmitted or received in proximity of a certain spot or area
H04L 51/52 - User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail for supporting social networking services
H04W 4/02 - Services making use of location information
H04W 4/021 - Services related to particular areas, e.g. point of interest [POI] services, venue services or geofences
H04W 4/029 - Location-based management or tracking services
H04W 4/18 - Information format or content conversion, e.g. adaptation by the network of the transmitted or received information for the purpose of wireless delivery to users or terminals
Disclosed is a method of receiving and processing content-sending inputs received by a head-worn device system including one or more display devices, one or more cameras and a vertically-arranged touchpad. The method includes displaying a content item on the one or more display devices, receiving a touch input on the touchpad corresponding to a send instruction, displaying a carousel of potential recipients, receiving a horizontal touch input on the touchpad, scrolling the carousel left or right on the one or more display devices in response to the horizontal touch input, receiving a tap touch input on the touchpad to select a particular recipient, receiving a further touch input, and in response to the further touch input, transmitting the content item to the selected recipient.
G06F 3/04815 - Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
G06F 3/0488 - Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
G06T 19/00 - Manipulating 3D models or images for computer graphics
Methods and systems are disclosed for performing operations for transferring garments from one real-world object to another in real time. The operations comprise receiving a first video that includes a depiction of a first person wearing a first upper-body garment in a first pose and obtaining a second video that includes a depiction of a second person wearing a second upper-body garment in a second pose. A pose of the second person depicted in the second video is modified to match the first pose of the first person depicted in the first video. The operations comprise generating an upper-body segmentation of the second upper-body garment which the second person is wearing in the second video in the modified pose and replacing the first upper-body garment worn by the first person in the first video with the second upper-body garment based on the upper-body segmentation.
An optical device for use in an augmented reality or virtual reality display, comprising: a waveguide; an input diffractive optical element, DOE, configured to receive light from a projector and to couple the received light into the waveguide along a plurality of optical paths; an output DOE offset from the input DOE along a first direction and configured to couple the received light out of the waveguide and towards a viewer; a first turning DOE offset from the input DOE along a second direction different from the first direction; wherein the input DOE is configured to couple a first portion of the received light in the second direction towards the first turning DOE and the first turning DOE is configured to diffract the first portion of the received light towards the output DOE, and the input DOE is configured to couple a second portion of the received light in the first direction towards the output DOE.
An eyewear device including a strain gauge sensor to determine when the eyewear device is manipulated by a user, such as being put on, taken off, and interacted with. A processor identifies a signature event based on sensor signals received from the strain gauge sensor and a data table of strain gauge sensor measurements corresponding to signature events. The processor controls the eyewear device as a function of the identified signature event, such as powering on a display of the eyewear device as the eyewear device is being put on a user's head, and then turning of the display when the eyewear device is removed from the user's head.
A waterproof UAV that records camera footage while traveling through air and while submerged in water. The UAV alters speed and direction of propellers dependent on the medium that the UAV is traveling through to provide control of the UAV. The propellers are capable of spinning in both directions to enable the UAV to change its depth and orientation in water. A machine learning (ML) model is used to identify humans and objects underwater. A housing coupled to the UAV makes the UAV positively buoyant to float in water and to control buoyancy while submerged.
Methods and systems are disclosed for performing real-time deforming operations. The system receives an image that includes a depiction of a real-world object. The system applies a machine learning model to the image to generate a warping field and segmentation mask, the machine learning model trained to establish a relationship between a plurality of training images depicting real -world objects and corresponding ground-truth warping fields and segmentation masks associated with a target shape. The system applies the generated warping field and segmentation mask to the image to warp the real- world object depicted in the image to the target shape.
A method for reducing motion-to-photon latency for hand tracking is described. In one aspect, a method includes accessing a first frame from a camera of an Augmented Reality (AR) device, tracking a first image of a hand in the first frame, rendering virtual content based on the tracking of the first image of the hand in the first frame, accessing a second frame from the camera before the rendering of the virtual content is completed, the second frame immediately following the first frame, tracking, using the computer vision engine of the AR device, a second image of the hand in the second frame, generating an annotation based on tracking the second image of the hand in the second frame, forming an annotated virtual content based on the annotation and the virtual content, and displaying the annotated virtual content in a display of the AR device.
Systems, methods, and computer readable media for graphical assistance with tasks using an augmented reality (AR) wearable devices are disclosed. Embodiments capture an image of a first user view of a real-world scene and access indications of surfaces and locations of the surfaces detected in the image. The AR wearable device displays indications of the surfaces on a display of the AR wearable device where the locations of the indications are based on the locations of the surfaces and a second user view of the real-world scene. The locations of the surfaces are indicated with 3D world coordinates. The user views are determined based on a location of the user. The AR wearable device enables a user to add graphics to the surfaces and select tasks to perform. Tools such as a bubble level or a measuring tool are available for the user to utilize to perform the task.
A content controller system to render frames on demand comprises a rendering server system that includes a plurality of graphics processing units (GPUs). The GPUs in the rendering server system render a set of media content item segments using a media content identification and a main user identification. Rendering the set of media content item segments includes retrieving metadata from a metadata database associated with the media content identification, rendering the set of media content item segments using the metadata, generating a main user avatar based on the main user identification, and incorporating the main user avatar into the set of media content item segments. The rendering server system then uploads the set of media content item segments to a segment database; and updates segment states in a segment state database to indicate that the set of media content item segments are available. Other embodiments are disclosed herein.
H04N 21/262 - Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission or generating play-lists
G06T 1/20 - Processor architectures; Processor configuration, e.g. pipelining
H04N 21/234 - Processing of video elementary streams, e.g. splicing of video streams or manipulating MPEG-4 scene graphs
H04N 21/235 - Processing of additional data, e.g. scrambling of additional data or processing content descriptors
H04N 21/239 - Interfacing the upstream path of the transmission network, e.g. prioritizing client requests
H04N 21/258 - Client or end-user data management, e.g. managing client capabilities, user preferences or demographics or processing of multiple end-users preferences to derive collaborative data
H04N 21/84 - Generation or processing of descriptive data, e.g. content descriptors
Various embodiments include systems, methods, and non-transitory computer-readable media for sharing and managing media galleries. Consistent with these embodiments, a method includes receiving a request from a first device to share a media gallery that includes a user avatar; generating metadata associated with the media gallery; generating a message associated with the media gallery, the message at least including the media gallery identifier and the identifier of the user avatar; and transmitting the message to a second device of the recipient user.
G06F 3/0482 - Interaction with lists of selectable items, e.g. menus
G06F 3/0484 - Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
H04L 67/146 - Markers for unambiguous identification of a particular session, e.g. session cookie or URL-encoding
A push notification mechanism at a mobile user device provides for automated limiting of the rate of production of push notification alerts (such as an audible alert or a vibratory alert) and/or push notifications responsive to the occurrence of chat events relevant to a chat application hosted by the user device. Some chat events automatically trigger suppression periods during which push notification alerts are prevented for subsequent chat events that satisfy predefined suppression criteria. Such push notification and/or alert limiting can be performed separately for separate users, chat groups, and/or chat event types.
H04L 51/04 - Real-time or near real-time messaging, e.g. instant messaging [IM]
H04L 51/224 - Monitoring or handling of messages providing notification on incoming messages, e.g. pushed notifications of received messages
H04L 51/52 - User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail for supporting social networking services
A three-dimensional asset (3D) reconstruction technique for generating a 3D asset representing an object from images of the object. The images are captured from different viewpoints in a darkroom using one or more light sources having known locations. The system estimates camera poses for each of the captured images and then constructs a 3D surface mesh made up of surfaces using the captured images and their respective estimated camera poses. Texture properties for each of the surfaces of the 3D surface mesh are then refined to generate the 3D asset.
A finger gesture recognition system is provided. The finger gesture recognition system includes one or more audio sensors and one or more optic sensors. The finger gesture recognition system captures, using the one or more audio sensors, audio signal data of a finger gesture being made by a user, and captures, using the one or more optic sensors, optic signal data of the finger gesture. The finger gesture recognition system recognizes the finger gesture based on the audio signal data and the optic signal data and communicates finger gesture data of the recognized finger gesture to an Augmented Reality/Combined Reality/Virtual Reality (XR) application.
A pose tracking system is provided. The pose tracking system includes an EMF tracking system having a user-worn head-mounted EMF source and one or more user-worn EMF tracking sensors attached to the wrists of the user. The EMF source is associated with a VIO tracking system such as AR glasses or the like. The pose tracking system determines a pose of the user's head and a ground plane using the VIO tracking system and a pose of the user's hands using the EMF tracking system to determine a fullbody pose for the user. Metal interference with the EMF tracking system is minimized using an IMU mounted with the EMF tracking sensors. Long term drift in the IMU and the VIO tracking system are minimized using the EMF tracking system.
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G06F 3/0346 - Pointing devices displaced or positioned by the user; Accessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
Eyewear devices including a tether and methods for identifying proper installation of the tether are disclosed. An eyewear device includes transmission lines extending through the temples to electrical and electronic components positioned adjacent to edges of a frame. A tether is attached to the temples to enable power and communication flow between the electrical and electronic components rather than through the frame. Proper installation is identified based on communications passing between the electrical and electronic components via the tether.
Methods and systems are disclosed for performing operations for controlling brightness in an AR device. The operations comprise displaying an image on an eyewear device worn by a user; detecting a gaze direction of a pupil of the user; identifying a first region of the image that corresponds to the gaze direction of the pupil; and modifying a brightness level or value of pixels in the image based on the gaze direction such that pixels in the first region of the image are set to a first brightness value and pixels in a second region of the image are set to a second brightness value that is lower than the first brightness value.
Methods and systems are disclosed for performing real-time deforming operations. The system receives an image that includes a depiction of a real-world object. The system applies a machine learning model to the image to generate a warping field and segmentation mask, the machine learning model trained to establish a relationship between a plurality of training images depicting real-world objects and corresponding ground-truth warping fields and segmentation masks associated with a target shape. The system applies the generated warping field and segmentation mask to the image to warp the real-world object depicted in the image to the target shape.
The subject technology detects from a set of frames, a first gesture, the first gesture corresponding to a pinch gesture. The subject technology detects a first location and a first position of a first representation of a first finger from the first gesture and a second location and a second position of a second representation of a second finger from the first gesture. The subject technology detects a first collision event corresponding to a first collider and a second collider intersecting with a third collider of a first virtual object. The subject technology detects a first change in the first location and the first position and a second change in the second location and the second position. The subject technology modifies the first virtual object to include an additional augmented reality content based at least in part on the first change and the second change.
Systems and methods are provided for clustering videos. The system accesses a plurality of content items, the plurality of content items comprising a first set of RGB video frames and a second set of optical flow frames corresponding to the first set of RGB video frames. The system processes the first set of RGB video frames by a first machine learning model to generate a first optimal assignment for the first set of RGB video frames, the first optimal assignment representing initial clustering of the first set of RGB video frames. The system generates an updated first optimal assignment for the first set of RGB video frames based on the first optimal assignment for the first set of RGB video frames and a second optimal assignment of the second set of optical flow frames, the second optimal assignment representing initial clustering of the second set of optical flow frames.
G06V 10/762 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
G06T 5/50 - Image enhancement or restoration by the use of more than one image, e.g. averaging, subtraction
G06V 10/82 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
95.
Virtual object manipulation with gestures in a messaging system
The subject technology detects a first gesture and a second gesture, each gesture corresponding to an open trigger finger gesture. The subject technology detects a third gesture and a fourth gesture, each gesture corresponding to a closed trigger finger gesture. The subject technology, selects a first virtual object in a first scene. The subject technology detects a first location and a first position of a first representation of a first finger from the third gesture and a second location and a second position of a second representation of a second finger from the fourth gesture. The subject technology detects a first change in the first location and the first position and a second change in the second location and the second position. The subject technology modifies a set of dimensions of the first virtual object to a different set of dimensions.
The subject technology detects a first location and a first position of a first representation of a first finger and a second location and a second position of a second representation of a second finger. The subject technology detects a first particular location and a first particular position of a first particular representation of a first particular finger and a second particular location and a second particular position of a second particular representation of a second particular finger. The subject technology detects a first change in the first location and the first position and a second change in the second location and the second position. The subject technology detects a first particular change in the first particular location and the first particular position and a second particular change in the second particular location and the second particular position. The subject technology generates a set of virtual objects.
In various embodiments, boundaries of geo-fences can be made mutable based on principles described herein. The term “mutable” refers to the ability of a thing (in this case, the boundary of a geo-fence) to change and adjust. In a typical embodiment, a mutable geo-fence system is configured to generate and monitor a geo-fence that encompasses a region, in order to dynamically vary the boundary of the geo-fence based on a number of boundary variables. The term “geo-fence” as used herein describes a virtual perimeter (e.g., a boundary) for a real-world geographic area. A geo-fence could be a radius around a point (e.g., a store), or a set of predefined boundaries. Boundary variables, as used herein, refers to a set of variables utilized by the mutable geo-fence system in determining a location of the boundary of the geo-fence.
Augmented reality guidance for guiding a user through an environment using an eyewear device. The eyewear device includes a display system and a position detection system. A user is guided though an environment by monitoring a current position of the eyewear device within the environment, identifying marker positions within a threshold of the current position, the marker positions defined with respect to the environment and associated with guidance markers, registering the marker positions, generating overlay image including the guidance markers, and presenting the overlay image on a display of the eyewear device.
Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for selecting ads for a video. The program and method provide for receiving a request for an ad to insert into a video playing on a client device, the request including a first content identifier that identifies a first type of content included in the video; determining a set of content identifiers associated with the first content identifier, the set of content identifiers identifying second types of content to filter with respect to providing the ad in response to the request; selecting an ad from among plural ads, by filtering ads tagged with a second content identifier included in the set of content identifiers; and providing the selected ad as a response to the request.
H04N 21/44 - Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to MPEG-4 scene graphs
H04N 21/4788 - Supplemental services, e.g. displaying phone caller identification or shopping application communicating with other users, e.g. chatting
A carry case for an electronics-enabled eyewear device, such as smart glasses, has charging contacts that are movable relative to a storage chamber in which the eyewear device is receivable. The charging contacts are connected to a battery carried by the case for charging the eyewear device via contact coupling of the charging contacts to corresponding contact formations on an exterior of the eyewear device. The charging contacts are in some instances mounted on respective flexible walls defining opposite extremities of the storage chamber. The contact formations on the eyewear device are in some instances provided by hinge assemblies that couple respective temples to a frame of the eyewear device.
