IMAGE COMPLETION WITH IMPROVED DEEP NEURAL NETWORKS

Digital image completion using deep learning is described. Initially, a digital image having at least one hole is received. This holey digital image is provided as input to an image completer formed with a framework that combines generative and discriminative neural networks based on learning architecture of the generative adversarial networks. From the holey digital image, the generative neural network generates a filled digital image having hole-filling content in place of holes. The discriminative neural networks detect whether the filled digital image and the hole filling digital content correspond to or include computer-generated content or are photo realistic. The generating and detecting are iteratively continued until the discriminative neural networks fail to detect computer-generated content for the filled digital image and hole-filling content or until detection surpasses a threshold difficulty. Responsive to this, the image completer outputs the filled digital image with hole-filling ...

JOINT DEPTH ESTIMATION AND SEMANTIC LABELING OF A SINGLE IMAGE

Abstract (57) A system and method of performing joint depth estimation and semantic labelling of an image by one or more computing devices, the method comprising the steps of: estimating global semantic and depth layouts of a scene of the image through machine learning by the one or more computing devices; estimating local semantic and depth layouts for respective ones of a plurality of segments of the scene of the image through machine learning by the one or more computing devices; and merging the estimated global semantic and depth layouts with the local semantic and depth layouts by the one or more computing devices to semantically label and assign a depth value to individual pixels in the image.

Automatically Selecting Example Stylized Images for Image Stylization Operations Based on Semantic Content

Systems and methods are provided for content-based selection of style examples used in image stylization operations. For example, training images can be used to identify example stylized images that will generate high-quality stylized images when stylizing input images having certain types of semantic content. In one example, a processing device determines which example stylized images are more suitable for use with certain types of semantic content represented by training images. In response to receiving or otherwise accessing an input image, the processing device analyzes the semantic content of the input image, matches the input image to at least one training image with similar semantic content, and selects at least one example stylized image that has been previously matched to one or more training images having that type of semantic content. The processing device modifies color or contrast information for the input image using the selected example stylized image. 1. A method for automatically selecting and applying an image stylization operation based on semantic content of a received image , the method comprising:determining that color information or contrast information of a training image is similar to color information or contrast information of an example stylized image;matching the training image to an input image that is semantically similar to the training image;selecting the example stylized image based on the example stylized image and the training image having similar color information or contrast information; andmodifying color information or contrast information of the input image based on the color information or contrast information from the selected example stylized image.2. The method of claim 1 , wherein the example stylized image is selected based on determining that the color information or contrast information of the example stylized image is similar to color information or contrast information of a sufficiently high number of images from a ...

Object detection via validation with visual search

One exemplary embodiment involves receiving, at a computing device comprising a processor, a test image having a candidate object and a set of object images detected to depict a similar object as the test image. The embodiment involves localizing the object depicted in each one of the object images based on the candidate object in the test image to determine a location of the object in each respective object image and then generating a validation score for the candidate object in the test image based at least in part on the determined location of the object in the respective object image and known location of the object in the same respective object image. The embodiment also involves computing a final detection score for the candidate object based on the validation score that indicates a confidence level that the object in the test image is located as indicated by the candidate object.

Gemeinsames Tiefenschätzen und semantisches Bezeichnen eines einzelnen Bildes

Es werden Techniken zum gemeinsamen Tiefenschätzen und semantisches Bezeichnen beschrieben, die verwendet werden können, um ein einzelnes Bild zu verarbeiten. In einer oder in mehreren Implementierungen werden globale semantische und Tiefenlayouts einer Szene des Bildes mittels maschinellem Lernen durch die eine oder die mehreren Rechnervorrichtungen geschätzt. Es werden ebenfalls lokale semantische und Tiefenlayouts für einzelne einer Vielzahl von Segmenten der Szene des Bildes mittels maschinellem Lernen durch die eine oder die mehreren Rechnervorrichtungen geschätzt. Die geschätzten globalen semantischen und Tiefenlayouts werden dann mit den lokalen semantischen und Tiefenlayouts zusammengeführt durch die eine oder die mehreren Rechnervorrichtungen, um einzelne Pixel in dem Bild semantisch zu bezeichnen und diesen einen Tiefenwert zuzuweisen.

Markieren großer Bilder unter Nutzung einer Bild-mit-Thema-Einbettung

Bereitgestellt wird ein Framework zum Verknüpfen von Bildern mit Themen unter Nutzung einbettenden Lernens. Das Framework wird unter Nutzung von Bildern trainiert, die jeweils mehrere visuelle Eigenschaften und mehrere damit verknüpfte Schlagwortmarkierungen aufweisen. Es werden visuelle Merkmale aus den visuellen Eigenschaften unter Nutzung eines faltungstechnischen neuronalen Netzwerkes berechnet, und es wird ein Bildmerkmalsvektor daraus erzeugt. Genutzt werden die Schlagwortmarkierungen zur Erzeugung eines gewichteten Wortvektors (oder eines „Soft-Topic-Merkmalsvektors“) für jedes Bild durch Berechnen eines gewichteten Durchschnittes von Wortvektordarstellungen, die die mit dem Bild verknüpften Schlagwortmarkierungen darstellen. Der Bildmerkmalsvektor und der Soft-Topic-Merkmalsvektor sind in einem gemeinsamen Einbettungsraum ausgerichtet, und es wird ein Relevanzkennwert für jede der Schlagwortmarkierungen berechnet. Sobald das Training erfolgt ist, kann das Framework Bilder automatisch ...

ATTRIBUTE RECOGNITION VIA VISUAL SEARCH

One exemplary embodiment involves identifying feature matches between each of a plurality of object images and a test image, each feature matches between a feature of a respective object image and a matching feature of the test image, wherein there is a spatial relationship between each respective object image feature and a test image feature, and wherein the object depicted in the test image comprises a plurality of attributes. Additionally, the embodiment involves estimating, for each attribute in the test image, an attribute value based at least in part on information stored in a metadata associated with each of the object images.

SEARCHING UNTAGGED IMAGES WITH TEXT-BASED QUERIES

In various implementations, a personal asset management application is configured to perform operations that facilitate the ability to search multiple images, irrespective of the images having characterizing tags associated therewith or without, based on a simple text-based query. A first search is conducted by processing a text-based query to produce a first set of result images used to further generate a visually-based query based on the first set of result images. A second search is conducted employing the visually-based query that was based on the first set of result images received in accordance with the first search conducted and based on the text-based query. The second search can generate a second set of result images, each having visual similarity to at least one of the images generated for the first set of result images. 1. A non-transitory computer storage medium storing computer-useable instructions that , when used by one or more computing devices , cause the one or more computing devices to perform operations comprising:receiving a text-based query for searching a first plurality of images, wherein each of at least some of the first plurality of images being associated with one or more characterizing tags;receiving, in accordance with a first search conducted based on the text-based query, a first set of result images from the first plurality of images, each result image in the first set having at least one characterizing tag corresponding to the text-based query;generating a visually-based query using one or more images from the first set of result images received in accordance with the first search conducted based on the text-based query, the visually-based query generated for searching at least one of: the first plurality of images and a second plurality of images; andreceiving, in accordance with a second search conducted based on the visually-based query, a second set of result images from the at least one of: the first plurality of images and the ...

Objektdetektion in Bildern

Bei Implementierungen einer Objektdetektion in Bildern werden Objektdetektoren unter Nutzung von heterogenen Trainingsdatensätzen trainiert. Ein erster Trainingsdatensatz wird dafür benutzt, ein Bildauszeichnungsnetzwerk zu trainieren, um eine Aufmerksamkeitskarte eines Eingabebildes für ein Zielkonzept zu bestimmen. Ein zweiter Trainingsdatensatz wird dafür benutzt, ein konditionales Detektionsnetzwerk zu trainieren, das als konditionale Eingaben die Aufmerksamkeitskarte und eine Worteinbettung des Zielkonzeptes annimmt. Obwohl das konditionale Detektionsnetzwerk mit einem Trainingsdatensatz trainiert wird, der eine kleine Anzahl von gesehenen Klassen (beispielsweis Klassen in einem Trainingsdatensatz) aufweist, verallgemeinert es durch Konzeptkonditionierung auf neuartige, ungesehene Klassen, da sich das Zielkonzept durch das konditionale Detektionsnetzwerk über die konditionalen Eingaben ausbreitet, was die Klassifikation und den Bereichsvorschlag beeinflusst. Klassen von Objekten, die ...

Image cropping suggestion using multiple saliency maps

Image cropping suggestion using multiple saliency maps is described. In one or more implementations, component scores, indicative of visual characteristics established for visually-pleasing croppings, are computed for candidate image croppings using multiple different saliency maps. The visual characteristics on which a candidate image cropping is scored may be indicative of its composition quality, an extent to which it preserves content appearing in the scene, and a simplicity of its boundary. Based on the component scores, the croppings may be ranked with regard to each of the visual characteristics. The rankings may be used to cluster the candidate croppings into groups of similar croppings, such that croppings in a group are different by less than a threshold amount and croppings in different groups are different by at least the threshold amount. Based on the clustering, croppings may then be chosen, e.g., to present them to a user for selection.

IMAGE COMPLETION WITH IMPROVED DEEP NEURAL NETWORKS

Predicting patch displacement maps using a neural network is described. Initially, a digital image on which an image editing operation is to be performed is provided as input to a patch matcher having an offset prediction neural network. From this image and based on the image editing operation for which this network is trained, the offset prediction neural network generates an offset prediction formed as a displacement map, which has offset vectors that represent a displacement of pixels of the digital image to different locations for performing the image editing operation. Pixel values of the digital image are copied to the image pixels affected by the operation by: determining the vectors pixels that correspond to the image pixels affected by the image editing operation and mapping the pixel values of the image pixels represented by the determined offset vectors to the affected pixels. According to this mapping, the pixel values of the affected pixels are set, effective to perform the ...

DEEP SALIENT OBJECT SEGMENTATION

Systems, methods, and non-transitory computer-readable media are disclosed for segmenting objects in digital visual media utilizing one or more salient content neural networks. In particular, in one or more embodiments, the disclosed systems and methods train one or more salient content neural networks to efficiently identify foreground pixels in digital visual media. Moreover, in one or more embodiments, the disclosed systems and methods provide a trained salient content neural network to a mobile device, allowing the mobile device to directly select salient objects in digital visual media utilizing a trained neural network. Furthermore, in one or more embodiments, the disclosed systems and methods train and provide multiple salient content neural networks, such that mobile devices can identify objects in real-time digital visual media feeds (utilizing a first salient content neural network) and identify objects in static digital images (utilizing a second salient content neural network ...

AESTHETICS-GUIDED IMAGE ENHANCEMENT

Methods and systems are provided for generating enhanced image. A neural network system is trained where the training includes training a first neural network that generates enhanced images conditioned on content of an image undergoing enhancement and training a second neural network that designates realism of the enhanced images generated by the first neural network. The neural network system is trained by determine loss and accordingly adjusting the appropriate neural network(s). The trained neural network system is used to generate an enhanced aesthetic image from a selected image where the output enhanced aesthetic image has increased aesthetics when compared to the selected image. 1. One or more computer-readable media having a plurality of executable instructions embodied thereon , which , when executed by one or more processors , cause the one or more processors to perform a method , the method comprising:obtaining scored images, wherein the images are scored based on aesthetic attributes;designating the scored images that have aesthetic scores within a predefined range as input images for training an image enhancement neural network of an aesthetic image enhancing neural network system;designating the scored images that have aesthetic scores above a predefined threshold as reference images for training an adversarial neural network of the aesthetic image enhancing neural network system; andtraining the aesthetic image enhancing neural network system using the input images and reference images, along with segmentation maps corresponding to the input images, wherein the trained aesthetic image enhancing neural network system is used to generate an enhanced image from an image input.2. The media of claim 1 , the method further comprising:outputting the enhanced image, wherein the enhanced image has increased aesthetics when compared with the image input into the trained aesthetic image enhancing neural network system.3. The media of claim 2 , the method further ...

DEEP SALIENT OBJECT SEGMENTATION

Systems, methods, and non-transitory computer-readable media are disclosed for segmenting objects in digital visual media utilizing one or more salient content neural networks. In particular, in one or more embodiments, the disclosed systems and methods train one or more salient content neural networks to efficiently identify foreground pixels in digital visual media. Moreover, in one or more embodiments, the disclosed systems and methods provide a trained salient content neural network to a mobile device, allowing the mobile device to directly select salient objects in digital visual media utilizing a trained neural network. Furthermore, in one or more embodiments, the disclosed systems and methods train and provide multiple salient content neural networks, such that mobile devices can identify objects in real-time digital visual media feeds (utilizing a first salient content neural network) and identify objects in static digital images (utilizing a second salient content neural network ...

METHOD FOR SCALING OBJECT DETECTION TO A VERY LARGE NUMBER OF CATEGORIES

In implementations of object detection in images, object detectors are trained using heterogeneous training datasets. A first training dataset is used to train an image tagging network to determine an attention map of an input image for a target concept. A second training dataset is used to train a conditional detection network that accepts as conditional inputs the attention map and a word embedding of the target concept. Despite the conditional detection network being trained with a training dataset having a small number of seen classes (e.g., classes in a training dataset), it generalizes to novel, unseen classes by concept conditioning, since the target concept propagates through the conditional detection network via the conditional inputs, thus influencing classification and region proposal. Hence, classes of objects that can be detected are expanded, without the need to scale training databases to include additional classes. Object Detection System 110 Object Detection Application ...

Method for optimizing vitrification ultra-low temperature preservation effect of lily embryonic callus

The invention discloses a method for optimizing a vitrification ultra-low temperature preservation effect of lily embryonic callus. According to the invention, the lily embryonic callus is processed by adopting a vitrification solution which contains a carbon nanometer material so as to improve the preservation effect of the lily embryonic callus. The method specifically comprises the steps of preculture, loading liquid treatment, vitrification solution treatment and liquid nitrogen preservation, wherein the vitrification solution contains 0.1-0.5 g/L of grapheme quantum dots. The method disclosed by the invention optimizes the preservation effect of the lily embryonic callus remarkably; as the grapheme quantum dots are added to serve as an allogenic material, the vitrification ultra-low temperature preservation of plants is facilitated.

Facilitating preservation of regions of interest in automatic image cropping

Embodiments of the present invention are directed to facilitating region of interest preservation. In accordance with some embodiments of the present invention, a region of interest preservation score using adaptive margins is determined. The region of interest preservation score indicates an extent to which at least one region of interest is preserved in a candidate image crop associated with an image. A region of interest positioning score is determined that indicates an extent to which a position of the at least one region of interest is preserved in the candidate image crop associated with the image. The region of interest preservation score and/or the preserving score are used to select a set of one or more candidate image crops as image crop suggestions.

Aesthetics-guided image enhancement

Methods and systems are provided for generating enhanced image. A neural network system is trained where the training includes training a first neural network that generates enhanced images conditioned on content of an image undergoing enhancement and training a second neural network that designates realism of the enhanced images generated by the first neural network. The neural network system is trained by determine loss and accordingly adjusting the appropriate neural network(s). The trained neural network system is used to generate an enhanced aesthetic image from a selected image where the output enhanced aesthetic image has increased aesthetics when compared to the selected image.

LARGE-SCALE IMAGE SEARCH AND TAGGING USING IMAGE-TO-TOPIC EMBEDDING

A framework is provided for associating images with topics utilizing embedding learning. The framework is trained utilizing images, each having multiple visual characteristics and multiple keyword tags associated therewith. Visual features are computed from the visual characteristics utilizing a convolutional neural network and an image feature vector is generated therefrom. The keyword tags are utilized to generate a weighted word vector (or "soft topic feature vector") for each image by calculating a weighted average of word vector representations that represent the keyword tags associated with the image. The image feature vector and the soft topic feature vector are aligned in a common embedding space and a relevancy score is computed for each of the keyword tags. Once trained, the framework can automatically tag images and a text based search engine can rank image relevance with respect to queried keywords based upon predicted relevancy scores. 10-USER 106- NETWORK 104 > If _------- ...

COMPOSITING AWARE IMAGE SEARCH

Compositing aware digital image search techniques and systems are described that leverage machine learning. In one example, a compositing aware image search system employs a two-stream convolutional neural network (CNN) to jointly learn feature embeddings from foreground digital images that capture a foreground object and background digital images that capture a background scene. In order to train models of the convolutional neural networks, triplets of training digital images are used. Each triplet may include a positive foreground digital image and a positive background digital image taken from the same digital image. The triplet also contains a negative foreground or background digital image that is dissimilar to the positive foreground or background digital image that is also included as part of the triplet. 1 Service Provider System 102 Compositing Aware Image Search System 118 122 / Background Feature Machine Learning System 124 Foreground Feature Machine LLearning System 126 Digital ...

Method for improving preservation effect of lily embryonic callus

The invention discloses a method for improving a preservation effect of lily embryonic callus. According to the invention, the lily embryonic callus is processed by adopting a vitrification solution which contains a carbon nanometer material so as to improve the preservation effect of the lily embryonic callus. The method specifically comprises the steps of pre-culture, contained liquid treatment, vitrification solution treatment and liquid nitrogen preservation, wherein the vitrification solution contains 0.1-0.5 g/L carbon nano-tubes. The method disclosed by the invention optimizes the preservation effect of the lily embryonic callus remarkably; as the carbon nano-tubes are added to serve as an allogenic material, the vitrification ultra-low temperature preservation of plants is facilitated.

Method for optimizing vitrified cryopreservation effect of agapanthus embryonic calluses

The invention discloses a method for optimizing a vitrified cryopreservation effect of agapanthus embryonic calluses. Cymbidium protocorms are processed by a vitrification solution containing carbon nanomaterials to improve the preservation effect. The method specifically comprises the following steps: preculturing; processing of a loading solution; processing of the vitrification solution; and preserving liquid nitrogen, wherein the vitrification solution contains 0.1-0.5g/L graphene quantum dots. According to the method disclosed by the invention, the preservation effect on the agapanthus embryonic calluses is significantly optimized, and the graphene quantum dots are added as an allogenic material to accelerate cryopreservation of plants by vitrification.

Video generation method, device and equipment

The invention provides a video generation method, device and equipment, and the method comprises the steps: obtaining a video generation request which comprises at least one initial image; obtaining a salient image and a drop image of the initial image; acquiring first image information of the salient image and second image information of the drop image; and according to the first image information and the second image information, determining a target mirror moving mode of the initial image, and according to the target mirror moving mode of the initial image, generating the video. And the video display effect is improved.

Feature Interpolation

Feature interpolation techniques are described. In a training stage, features are extracted from a collection of training images and quantized into visual words. Spatial configurations of the visual words in the training images are determined and stored in a spatial configuration database. In an object detection stage, a portion of features of an image are extracted from the image and quantized into visual words. Then, a remaining portion of the features of the image are interpolated using the visual words and the spatial configurations of visual words stored in the spatial configuration database. 1. A computer-implemented method comprising:receiving a collection of training images;extracting features from the collection of training images;quantizing the extracted features into visual words;determining spatial configurations of the visual words in the training images; andstoring the spatial configurations of the visual words in a spatial configuration database.2. The computer-implemented method of claim 1 , wherein the determining the spatial configurations of the visual words comprises claim 1 , for each visual word claim 1 , determining neighboring visual words.3. The computer-implemented method of claim 2 , wherein the storing the spatial configurations of the visual words comprises claim 2 , for each visual word claim 2 , storing the neighboring visual words in neighbor lists that are associated with the visual word in the spatial configuration database.4. The computer-implemented method of claim 3 , wherein each neighbor list includes an occurrence field that indicates a number of occurrences of the neighboring visual word in the training images.5. The computer-implemented method of claim 4 , wherein the storing the neighboring visual words in the neighbor lists further comprises claim 4 , for each neighboring visual word:determining if the neighboring visual word is listed in a corresponding neighbor list of the visual word;if the neighboring visual word is ...

METHOD FOR SCALING OBJECT DETECTION TO A VERY LARGE NUMBER OF CATEGORIES

In implementations of object detection in images, object detectors are trained using heterogeneous training datasets. A first training dataset is used to train an image tagging network to determine an attention map of an input image for a target concept. A second training dataset is used to train a conditional detection network that accepts as conditional inputs the attention map and a word embedding of the target concept. Despite the conditional detection network being trained with a training dataset having a small number of seen classes (e.g., classes in a training dataset), it generalizes to novel, unseen classes by concept conditioning, since the target concept propagates through the conditional detection network via the conditional inputs, thus influencing classification and region proposal. Hence, classes of objects that can be detected are expanded, without the need to scale training databases to include additional classes. Object Detection System 110 Object Detection Application ...

Utilizing deep learning for boundary-aware image segmentation

Systems and methods are disclosed for segmenting a digital image to identify an object portrayed in the digital image from background pixels in the digital image. In particular, in one or more embodiments, the disclosed systems and methods use a first neural network and a second neural network to generate image information used to generate a segmentation mask that corresponds to the object portrayed in the digital image. Specifically, in one or more embodiments, the disclosed systems and methods optimize a fit between a mask boundary of the segmentation mask to edges of the object portrayed in the digital image to accurately segment the object within the digital image.

JOINT DEPTH ESTIMATION AND SEMANTIC LABELING OF A SINGLE IMAGE

Abstract (57) A system and method of performing joint depth estimation and semantic labelling of an image by one or more computing devices, the method comprising the steps of: estimating global semantic and depth layouts of a scene of the image through machine learning by the one or more computing devices; estimating local semantic and depth layouts for respective ones of a plurality of segments of the scene of the image through machine learning by the one or more computing devices; and merging the estimated global semantic and depth layouts with the local semantic and depth layouts by the one or more computing devices to semantically label and assign a depth value to individual pixels in the image.

PHRASECUT: SEGMENTATION USING NATURAL LANGUAGE INPUTS

The invention is directed towards segmenting images based on natural language phrases. An image and an n-gram, including a sequence of tokens, are received. An encoding of image features and a sequence of token vectors are generated. A fully convolutional neural network identifies and encodes the image features. A word embedding model generates the token vectors. A recurrent neural network (RNN) iteratively updates a segmentation map based on combinations of the image feature encoding and the token vectors. The segmentation map identifies which pixels are included in an image region referenced by the n-gram. A segmented image is generated based on the segmentation map. The RNN may be a convolutional multimodal RNN. A separate RNN, such as a long short-term memory network, may iteratively update an encoding of semantic features based on the order of tokens. The first RNN may update the segmentation map based on the semantic feature encoding. C:0 - 2 (U m OD . 0 0 m M0 Er)c 0) 1 W0 LEQ)-a ...

DENSE IMAGE TAGGING VIA TWO-STAGE SOFT TOPIC EMBEDDING

A framework is provided for associating dense images with topics. The framework is trained utilizing images, each having multiple regions, multiple visual characteristics and multiple keyword tags associated therewith. For each region of each image, visual features are computed from the visual characteristics utilizing a convolutional neural network, and an image feature vector is generated from the visual features. The keyword tags are utilized to generate a weighted word vector for each image by calculating a weighted average of word vector representations representing the keyword tags associated with the image. The image feature vector and the weighted word vector are aligned in a common embedding space and a heat map is computed for the image. Once trained, the framework can be utilized to automatically tag images and rank the relevance of images with respect to queried keywords based upon associated heat maps. 102 USERC DEVICE] 106 NETWORK 104>_ _ IMAGE EMBEDDING SYSTEM 110 IMAGE TAG ...

Automatically segmenting images based on natural language phrases

The invention is directed towards segmenting images based on natural language phrases. An image and an n-gram, including a sequence of tokens, are received. An encoding of image features and a sequence of token vectors are generated. A fully convolutional neural network identifies and encodes the image features. A word embedding model generates the token vectors. A recurrent neural network (RNN) iteratively updates a segmentation map based on combinations of the image feature encoding and the token vectors. The segmentation map identifies which pixels are included in an image region referenced by the n-gram. A segmented image is generated based on the segmentation map. The RNN may be a convolutional multimodal RNN. A separate RNN, such as a long short-term memory network, may iteratively update an encoding of semantic features based on the order of tokens. The first RNN may update the segmentation map based on the semantic feature encoding.

CONCEPT MASK: LARGE-SCALE SEGMENTATION FROM SEMANTIC CONCEPTS

Semantic segmentation techniques and systems are described that overcome the challenges of limited availability of training data to describe the potentially millions of tags that may be used to describe semantic classes in digital images. In one example, the techniques are configured to train neural networks to leverage different types of training datasets using sequential neural networks and use of vector representations to represent the different semantic classes. Computing Device 102 Image Processing System 104 Semantic Digital Image Segmentation Class 120System116 Indication 122 Attention Map 24- Network (1rk) Digital Image 106 ?74 1 ...

Video generation method and device based on music points, equipment and storage medium

The invention provides a video generation method and device based on music points, equipment and a storage medium. The method comprises the following steps: respectively obtaining a plurality of video objects and audio information; determining a plurality of initial music points in the audio information and feature information of each initial music point, wherein the feature information at least comprises the sound intensity of each initial music point and the moment of each initial music point in the audio information; screening out a target music point from the plurality of initial music points according to the feature information; and generating a target video according to the target music point and the plurality of video objects. The video generation method and device based on the music points, the equipment and the storage medium provided by the invention are used for improving the richness of the target video.

BOARD HARDWARE DEVICE AND RADIO FREQUENCY BLIND-MATE CONNECTION DEVICE

An RF blind-mate connection device disclosed herein includes a duplexer, a power amplification circuit board, a transceiver, an RF signal connector, and a power connector. The duplexer and the transceiver are located at one end of the RF blind-mate connection device, and the transceiver is fixed on the duplexer; the power amplification circuit board is located at the other end of the RF blind-mate connection device, and the location of the power amplification circuit board corresponds to that of the duplexer; the RF signal connector is fixed on the duplexer and the power amplification circuit board; the power connector is fixed on the transceiver and the power amplification circuit board; and the RF signal connector and the power connector transmit both the power signal and the RF signal in a blind-mate way. A board hardware device is disclosed herein to transmit RF signals and power signals inside the RF module through the connector.

Joint training technique for depth map generation

Joint training technique for depth map generation implemented by depth prediction system as part of a computing device is described. The depth prediction system is configured to generate a candidate feature map from features extracted from training digital images, generate a candidate segmentation map and a candidate depth map from the generated candidate feature map, and jointly train portions of the depth prediction system using a loss function. Consequently, depth prediction system is able to generate a depth map that identifies depths of objects using ordinal depth information and accurately delineates object boundaries within a single digital image.

JOINT BLUR MAP ESTIMATION AND BLUR DESIRABILITY CLASSIFICATION FROM AN IMAGE

Techniques are disclosed for blur classification. The techniques utilize an image content feature map, a blur map, and an attention map, thereby combining low-level blur estimation with a high-level understanding of important image content in order to perform blur classification. The techniques allow for programmatically determining if blur exists in an image, and determining what type of blur it is (e.g., high blur, low blur, middle or neutral blur, or no blur). According to one example embodiment, if blur is detected, an estimate of spatially-varying blur amounts is performed and blur desirability is categorized in terms of image quality.

Method and Apparatus for Determining Motion

An apparatus, comprising a processor and memory configured to cause the apparatus to perform at least the following: receiving a video indicating a motion, generating a set of normalized representations of movement based, at least in part, on the video, evaluating a reference set of representations with respect to the set of normalized representations of the movement, and determining that at least one predetermined motion correlates to the set of normalized representations of the movement based, at least in part, on the evaluation is disclosed. 1. An apparatus , comprising:a processor;memory including computer program code, the memory and the computer program code configured to, working with the processor, cause the apparatus to perform at least the following:receiving a video indicating a motion;generating a set of normalized representations of movement based, at least in part, on the video;evaluating a reference set of representations with respect to the set of normalized representations of the movement; anddetermining that at least one predetermined motion correlates to the set of normalized representations of the movement based, at least in part, on the evaluation.2. The apparatus of claim 1 , wherein generating the set of normalized representations of the movement comprises determining a video time interval claim 1 , and evaluating information from the video based claim 1 , at least in part claim 1 , on the video time interval.3. The apparatus of claim 2 , wherein the video time interval relates to a time interval that substantially uniformly segments at least part of the video.4. The apparatus of claim 2 , wherein the video time interval is based claim 2 , at least in part claim 2 , on number of representations of the reference set of representations that are associated with the predetermined motion.5. The apparatus of claim 1 , wherein the set of normalized representations of movement has a normalized structure that comprises a predetermined number of local ...

A single-board hardware device and RF blind insertion connection device

Utilizing a deep neural network-based model to identify visually similar digital images based on user-selected visual attributes

The present disclosure relates to systems, methods, and non-transitory computer readable media for utilizing a deep neural network-based model to identify similar digital images for query digital images. For example, the disclosed systems utilize a deep neural network-based model to analyze query digital images to generate deep neural network-based representations of the query digital images. In addition, the disclosed systems can generate results of visually-similar digital images for the query digital images based on comparing the deep neural network-based representations with representations of candidate digital images. Furthermore, the disclosed systems can identify visually similar digital images based on user-defined attributes and image masks to emphasize specific attributes or portions of query digital images.

Object Detection In Images

In implementations of object detection in images, object detectors are trained using heterogeneous training datasets. A first training dataset is used to train an image tagging network to determine an attention map of an input image for a target concept. A second training dataset is used to train a conditional detection network that accepts as conditional inputs the attention map and a word embedding of the target concept. Despite the conditional detection network being trained with a training dataset having a small number of seen classes (e.g., classes in a training dataset), it generalizes to novel, unseen classes by concept conditioning, since the target concept propagates through the conditional detection network via the conditional inputs, thus influencing classification and region proposal. Hence, classes of objects that can be detected are expanded, without the need to scale training databases to include additional classes. 1. In a digital medium environment for detecting objects in digital images , a method implemented by a computing device , the method comprising:receiving a word-based concept and an input digital image that depicts a scene;generating a word embedding based on the word-based concept, the word embedding describing relationships between the word-based concept and different word-based concepts; andgenerating an output digital image from the input digital image and the word embedding, the output digital image depicting the scene and including one or more bounding containers that denote regions of the scene that include one or more objects corresponding to the word-based concept.2. The method as described in claim 1 , wherein the generating the output digital image includes generating the output digital image with a conditional detection network of the computing device that is trained to detect the objects in the digital images that do not include the one or more objects corresponding to the word-based concept.3. The method as described in claim ...

PHRASECUT: SEGMENTATION USING NATURAL LANGUAGE INPUTS

The invention is directed towards segmenting images based on natural language phrases. An image and an n-gram, including a sequence of tokens, are received. An encoding of image features and a sequence of token vectors are generated. A fully convolutional neural network identifies and encodes the image features. A word embedding model generates the token vectors. A recurrent neural network (RNN) iteratively updates a segmentation map based on combinations of the image feature encoding and the token vectors. The segmentation map identifies which pixels are included in an image region referenced by the n-gram. A segmented image is generated based on the segmentation map. The RNN may be a convolutional multimodal RNN. A separate RNN, such as a long short-term memory network, may iteratively update an encoding of semantic features based on the order of tokens. The first RNN may update the segmentation map based on the semantic feature encoding. 0) U)( C'a C :3 0 E rr E a) u o 2! E ( W 6 cCU ...

DENSE IMAGE TAGGING VIA TWO-STAGE SOFT TOPIC EMBEDDING

A framework is provided for associating dense images with topics. The framework is trained utilizing images, each having multiple regions, multiple visual characteristics and multiple keyword tags associated therewith. For each region of each image, visual features are computed from the visual characteristics utilizing a convolutional neural network, and an image feature vector is generated from the visual features. The keyword tags are utilized to generate a weighted word vector for each image by calculating a weighted average of word vector representations representing the keyword tags associated with the image. The image feature vector and the weighted word vector are aligned in a common embedding space and a heat map is computed for the image. Once trained, the framework can be utilized to automatically tag images and rank the relevance of images with respect to queried keywords based upon associated heat maps. 102 USERC DEVICE] 100 106 NETWORK 104>_ _ IMAGE EMBEDDING SYSTEM 110 IMAGE ...

SELECTIVE AND MULTI-QUERY VISUAL SIMILARITY SEARCH

The present disclosure relates to systems, methods, and non-transitory computer readable media for utilizing a deep neural network-based model to identify similar digital images for query digital images. For example, the disclosed systems utilize a deep neural network-based model to analyze query digital images to generate deep neural network-based representations of the query digital images. In addition, the disclosed systems can generate results of visually-similar digital images for the query digital images based on comparing the deep neural network-based representations with representations of candidate digital images. Furthermore, the disclosed systems can identify visually similar digital images based on user defined attributes and image masks to emphasize specific attributes or portions of query digital images. =34 o0 oC E u V 0 4-, UD ...

Joint blur map estimation and blur desirability classification from an image

Techniques are disclosed for blur classification. The techniques utilize an image content feature map, a blur map, and an attention map, thereby combining low-level blur estimation with a high-level understanding of important image content in order to perform blur classification. The techniques allow for programmatically determining if blur exists in an image, and determining what type of blur it is (e.g., high blur, low blur, middle or neutral blur, or no blur). According to one example embodiment, if blur is detected, an estimate of spatially-varying blur amounts is performed and blur desirability is categorized in terms of image quality.

Saliency map computation

Saliency map computation is described. In one or more implementations, a base saliency map is generated for an image of a scene. The base saliency map may be generated from intermediate saliency maps computed for boundary regions of the image. Each of the intermediate saliency maps may represent visual saliency of portions of the scene that are captured in the corresponding boundary region. The boundary regions may include, for instance, a top boundary region, a bottom boundary region, a left boundary region, and a right boundary region. Further, the intermediate saliency maps may be combined in such a way that an effect of a foreground object on the saliency map is suppressed. The foreground objects for which the effect is suppressed are those that occupy a majority of one of the boundary regions.

Planar region guided 3D geometry estimation from a single image

Techniques for planar region-guided estimates of 3D geometry of objects depicted in a single 2D image. The techniques estimate regions of an image that are part of planar regions (i.e., flat surfaces) and use those planar region estimates to estimate the 3D geometry of the objects in the image. The planar regions and resulting 3D geometry are estimated using only a single 2D image of the objects. Training data from images of other objects is used to train a CNN with a model that is then used to make planar region estimates using a single 2D image. The planar region estimates, in one example, are based on estimates of planarity (surface plane information) and estimates of edges (depth discontinuities and edges between surface planes) that are estimated using models trained using images of other scenes.

ACCURATE TAG RELEVANCE PREDICTION FOR IMAGE SEARCH

Embodiments of the present invention provide an automated image tagging system that can predict a set of tags, along with relevance scores, that can be used for keyword-based image retrieval, image tag proposal, and image tag auto-completion based on user input. Initially, during training, a clustering technique is utilized to reduce cluster imbalance in the data that is input into a convolutional neural network (CNN) for training feature data. In embodiments, the clustering technique can also be utilized to compute data point similarity that can be utilized for tag propagation (to tag untagged images). During testing, a diversity based voting framework is utilized to overcome user tagging biases. In some embodiments, bigram re-weighting can down-weight a keyword that is likely to be part of a bigram based on a predicted tag set. 1. A computer-implemented method for training classifiers to tag images , the method comprising:receiving a set of input data including images and corresponding image tags;partitioning the set of input data into a first cluster of data, a second cluster of data, and a third cluster of data based on similarity of the images, wherein the first cluster of data includes a first set of images and corresponding image tags, the first set of images being similar to one another, the second cluster of data includes a second set of images and corresponding image tags, the second set of images being similar to one another, and the third cluster of data includes a third set of images and corresponding image tags, the third set of images being similar to one another;determining that a size of the first cluster of data is less than a predefined threshold and that a size of the second cluster of data and a size of the third cluster of data exceed the predefined threshold;based on the size of the second cluster of data and the size of the third cluster of data exceeding the predefined threshold, partitioning the union of the second and third sets of images ...

Genaues Vorhersagen einer Etikettrelevanz bei einer Bildabfrage

Bereitgestellt wird durch Ausführungsformen der vorliegenden Erfindung ein automatisiertes Bildetikettierungssystem zur Vorhersage eines Satzes von Etiketten zusammen mit Relevanzkennwerten zur Verwendung für einen schlüsselwortbasierten Bildabruf, einen Bildetikettvorschlag und eine Bildetikettautovervollständigung auf Grundlage einer Nutzereingabe. Zu Beginn wird beim Training eine Clusterungstechnik genutzt, um eine Clusterunausgewogenheit bei den Daten, die in ein faltungstechnisches neuronales Netzwerk (CNN) zum Trainieren von Merkmalsdaten eingegeben werden, zu verringern. Bei Ausführungsformen kann die Clusterungstechnik auch genutzt werden, um eine Datenpunktähnlichkeit, die zur Etikettübertragung (zum Etikettieren von nicht etikettierten Bildern) genutzt werden kann, zu berechnen. Beim Testen wird ein diversitätsbasiertes Wahlstimmensystem genutzt, um Nutzeretikettierungsvorlieben zu überwinden. Bei einigen Ausführungsformen kann ein Bigrammneugewichten ein Schlüsselwort, das wahrscheinlich ...

COMPOSITING AWARE IMAGE SEARCH

Compositing aware digital image search techniques and systems are described that leverage machine learning. In one example, a compositing aware image search system employs a two-stream convolutional neural network (CNN) to jointly learn feature embeddings from foreground digital images that capture a foreground object and background digital images that capture a background scene. In order to train models of the convolutional neural networks, triplets of training digital images are used. Each triplet may include a positive foreground digital image and a positive background digital image taken from the same digital image. The triplet also contains a negative foreground or background digital image that is dissimilar to the positive foreground or background digital image that is also included as part of the triplet. 1 Service Provider System 102 Compositing Aware Image Search System 118 122 / Background Feature Machine Learning System 124 Foreground Feature Machine LLearning System 126 Digital ...

LARGE-SCALE IMAGE SEARCH AND TAGGING USING IMAGE-TO-TOPIC EMBEDDING

A framework is provided for associating images with topics utilizing embedding learning. The framework is trained utilizing images, each having multiple visual characteristics and multiple keyword tags associated therewith. Visual features are computed from the visual characteristics utilizing a convolutional neural network and an image feature vector is generated therefrom. The keyword tags are utilized to generate a weighted word vector (or "soft topic feature vector") for each image by calculating a weighted average of word vector representations that represent the keyword tags associated with the image. The image feature vector and the soft topic feature vector are aligned in a common embedding space and a relevancy score is computed for each of the keyword tags. Once trained, the framework can automatically tag images and a text based search engine can rank image relevance with respect to queried keywords based upon predicted relevancy scores. 10-USER 106- NETWORK 104 > If _------- ...

IMAGE COMPLETION WITH IMPROVED DEEP NEURAL NETWORKS

Predicting patch displacement maps using a neural network is described. Initially, a digital image on which an image editing operation is to be performed is provided as input to a patch matcher having an offset prediction neural network. From this image and based on the image editing operation for which this network is trained, the offset prediction neural network generates an offset prediction formed as a displacement map, which has offset vectors that represent a displacement of pixels of the digital image to different locations for performing the image editing operation. Pixel values of the digital image are copied to the image pixels affected by the operation by: determining the vectors pixels that correspond to the image pixels affected by the image editing operation and mapping the pixel values of the image pixels represented by the determined offset vectors to the affected pixels. According to this mapping, the pixel values of the affected pixels are set, effective to perform the ...

Integration lamp pole basic station

The utility model relates to a basic station technical field that builds a station especially relates to an integration lamp pole basic station. Can be in the same place basic station and the integration of the lamp pole body of rod to can effectively help the operator to acquire the site in batches, realize building a station fast. The utility model provides an integration lamp pole basic stationincludes: the lamp pole body of rod and basic station, wherein, this basic station includes the baseband processing unit, a power supply, a battery, the radio frequency, antenna and be used for connecting this baseband processing unit and this radio optical transmission module, it holds the chamber to be provided with at least one in this lamp pole body of rod, this lamp pole body of rod enclosesto establish and is equipped with the opening on this casing that holds the chamber, this opening part is equipped with chamber door, this baseband processing unit, power and this opening of batteryaccessible ...

Personalized Digital Image Aesthetics in a Digital Medium Environment

Techniques and systems are described to determine personalized digital image aesthetics in a digital medium environment. In one example, a personalized offset is generated to adapt a generic model for digital image aesthetics. A generic model, once trained, is used to generate training aesthetics scores from a personal training data set that corresponds to an entity, e.g., a particular user, group of users, and so on. The image aesthetics system then generates residual scores (e.g., offsets) as a difference between the training aesthetics score and the personal aesthetics score for the personal training digital images. The image aesthetics system then employs machine learning to train a personalized model to predict the residual scores as a personalized offset using the residual scores and personal training digital images. 1. In a digital medium environment for personalized digital image aesthetics , a method implemented by at least one computing device , the method comprising:generating, by the at least one computing device, a generic aesthetics score for a digital image using at least one generic model trained using machine learning on a generic training data set;generating, by the at least one computing device, a personalized offset for the digital image using at least one personalized model trained using machine learning on a personal training data set associated with an entity;determining, by the at least one computing device, a personalized aesthetics score for the digital image based on the personalized offset and the generic aesthetics score; andoutputting, by the at least one computing device, the personalized aesthetics score for the digital image.2. The method as described in claim 1 , wherein:the at least one personalized model is trained using machine learning based on a deviation of a personal aesthetics score with respect to a training aesthetics score;the personal aesthetics score is specified by the entity as part of the personal training data set; ...

Image crop suggestion and evaluation using deep-learning

Various embodiments describe using a neural network to evaluate image crops in substantially real-time. In an example, a computer system performs unsupervised training of a first neural network based on unannotated image crops, followed by a supervised training of the first neural network based on annotated image crops. Once this first neural network is trained, the computer system inputs image crops generated from images to this trained network and receives composition scores therefrom. The computer system performs supervised training of a second neural network based on the images and the composition scores.

Image Depth Inference from Semantic Labels

Image depth inference techniques and systems from semantic labels are described. In one or more implementations, a digital medium environment includes one or more computing devices to control a determination of depth within an image. Regions of the image are semantically labeled by the one or more computing devices. At least one of the semantically labeled regions is decomposed into a plurality of segments formed as planes generally perpendicular to a ground plane of the image. Depth of one or more of the plurality of segments is then inferred based on relationships of respective segments with respective locations of the ground plane of the image. A depth map is formed that describes depth for the at least one semantically labeled region based at least in part on the inferred depths for the one or more of the plurality of segments. 1. In a digital medium environment including one or more computing devices to control a determination of depth within an image , a method comprising:decomposing at least one of a plurality of semantically labeled regions of an image by the one or more computing devices into a plurality of segments formed as planes that are generally perpendicular to a ground plane of the image;inferring depth of one or more of the plurality of segments by the one or more computing devices based on relationship of respective said segments with respective locations of the ground plane of the image; andforming a depth map by the one or more computing devices that describes depth for the at least one semantically labeled region based at least in part on the inferred depths for the one or more of plurality of segments.2. The method as described in claim 1 , wherein semantic labels used in the semantic labeling are selected from a plurality of categories.3. The method as described in claim 2 , wherein the plurality of categories include sky claim 2 , ground claim 2 , building claim 2 , plant claim 2 , or object.4. The method as described in claim 1 , wherein ...

Themenverknüpfung und Markierung für dichte Bilder

Bereitgestellt wird ein Framework zum Verknüpfen dichter Bilder mit Themen. Das Framework wird unter Nutzung von Bildern trainiert, die jeweils mehrere Bereiche, mehrere visuelle Eigenschaften und mehrere Schlagwortmarkierungen, die damit verknüpft sind, aufweisen. Für jeden Bereich eines jeden Bildes werden visuelle Merkmale aus den visuellen Eigenschaften unter Nutzung eines faltungstechnischen neuronalen Netzwerkes berechnet, und es wird ein Bildmerkmalsvektor aus den visuellen Merkmalen erzeugt. Genutzt werden die Schlagwortmarkierungen zur Erzeugung eines gewichteten Wortvektors für jedes Bild durch Berechnen eines gewichteten Durchschnittes von Wortvektordarstellungen, die die mit dem Bild verknüpften Schlagwortmarkierungen darstellen. Der Bildmerkmalsvektor und der gewichtete Wortvektor werden in einem gemeinsamen Einbettungsraum ausgerichtet, und es wird eine Wärmekarte für das Bild berechnet. Sobald das Training erfolgt ist, kann das Framework dafür genutzt werden, Bilder automatisch ...

Automatisches Segmentieren von Bildern auf Grundlage von in natürlicher Sprache gegebenen Äußerungen

Die Erfindung betrifft das Segmentieren von Bildern auf Grundlage von in natürlicher Sprache gegebenen Äußerungen. Es werden ein Bild und ein n-Gramm mit einer Sequenz von Tokens empfangen. Es werden eine Codierung von Bildmerkmalen und eine Sequenz von Tokenvektoren erzeugt. Ein vollständig faltungstechnisches neuronales Netzwerk identifiziert und codiert die Bildmerkmale. Ein worteinbettendes Modell erzeugt die Tokenvektoren. Ein rekurrentes neuronales Netzwerk (RNN) aktualisiert iterativ eine Segmentierungsabbildung auf Grundlage von Kombinationen der Bildmerkmalscodierung und der Tokenvektoren. Die Segmentierungsabbildung identifiziert, welche Pixel in einem Bildbereich beinhaltet sind, auf den von dem n-Gramm verwiesen wird. Es wird ein segmentiertes Bild auf Grundlage der Segmentierungsabbildung erzeugt. Das RNN kann ein faltungstechnisches multimodales RNN sein. Ein getrenntes RNN, so beispielsweise ein LSTM-Netzwerk (Long Short-Term Memory LSTM), kann iterativ eine Codierung von ...

ACCURATE TAG RELEVANCE PREDICTION FOR IMAGE SEARCH

A computer-implemented method for training classifiers to tag images is provided. A set of input data including images and corresponding image tags is received. The received data is partitioned into first and second clusters of data based on similarity of the images. After determining that a size of the first cluster exceeds a predetermined threshold and the size of the second cluster is less than the threshold, the first cluster of data is portioned into third and fourth clusters of data, the size of the third and fourth clusters being less than the threshold. A classifier is trained to predict image tags for an untagged image using the second, third and fourth clusters of data. DATABASE USER DATABASE 102IC NETWORK 106B DATABASE DVC TAGGING ENGINE 112, TRAINING COMPONENT 1141 PROPAGATION COMPONENT FIG.1 1161 PREDICTION COMPONENT ...

ACCURATE TAG RELEVANCE PREDICTION FOR IMAGE SEARCH

A computer-implemented method for training classifiers to tag images is provided. A set of input data including images and corresponding image tags is received. The received data is partitioned into first and second clusters of data based on similarity of the images. After determining that a size of the first cluster exceeds a predetermined threshold and the size of the second cluster is less than the threshold, the first cluster of data is portioned into third and fourth clusters of data, the size of the third and fourth clusters being less than the threshold. A classifier is trained to predict image tags for an untagged image using the second, third and fourth clusters of data. DATABASE USER DATABASE TA DEVICE NETWORK 106B DATABASE DVC TAGGING ENGINE 112, TRAINING COMPONENT 114, PROPAGATION COMPONENT FIG. 1 116- PREDICTION COMPONENT ...

CONCEPT MASK: LARGE-SCALE SEGMENTATION FROM SEMANTIC CONCEPTS

Semantic segmentation techniques and systems are described that overcome the challenges of limited availability of training data to describe the potentially millions of tags that may be used to describe semantic classes in digital images. In one example, the techniques are configured to train neural networks to leverage different types of training datasets using sequential neural networks and use of vector representations to represent the different semantic classes. Computing Device 102 Image Processing System 104 Semantic Digital Image Segmentation Class 120System116 Indication 122 Attention Map 24- Network (1rk) Digital Image 106 ?74 1 ...

SELECTIVE AND MULTI-QUERY VISUAL SIMILARITY SEARCH

The present disclosure relates to systems, methods, and non-transitory computer readable media for utilizing a deep neural network-based model to identify similar digital images for query digital images. For example, the disclosed systems utilize a deep neural network-based model to analyze query digital images to generate deep neural network-based representations of the query digital images. In addition, the disclosed systems can generate results of visually-similar digital images for the query digital images based on comparing the deep neural network-based representations with representations of candidate digital images. Furthermore, the disclosed systems can identify visually similar digital images based on user defined attributes and image masks to emphasize specific attributes or portions of query digital images. =34 o0 oC E u V 0 4-, UD ...

IMAGE-BLENDING VIA ALIGNMENT OR PHOTOMETRIC ADJUSTMENTS COMPUTED BY A NEURAL NETWORK

Certain embodiments involve blending images using neural networks to automatically generate alignment or photometric adjustments that control image blending operations. For instance, a foreground image and a background image data are provided to an adjustment-prediction network that has been trained, using a reward network, to compute alignment or photometric adjustments that optimize blending reward scores. An adjustment action (e.g., an alignment or photometric adjustment) is computed by applying the adjustment-prediction network to the foreground image and the background image data. A target background region is extracted from the background image data by applying the adjustment action to the background image data. The target background region is blended with the foreground image, and the resultant blended image is outputted. 1. A method for blending images using alignment or photometric adjustments computed by a neural network , wherein the method includes one or more processing devices performing operations comprising:accessing, from a memory device, a foreground image and background image data that includes or is computed from a background image; (i) the adjustment-prediction network is trained, with a reward network, to compute one or more of training alignment adjustments and training photometric adjustments that optimize a training blending reward score,', '(ii) the training blending reward score is computed by applying the reward network to an image blending result, and', '(ii) the image blending result is generated by blending a training foreground image with a training background image having the one or more of the training alignment adjustments and the training photometric adjustments;, 'providing the foreground image and the background image data to an adjustment-prediction network, whereincomputing an adjustment action by applying the adjustment-prediction network to the foreground image and the background image data, the adjustment action comprising ...

Zusammenstellungssensitive Digitalbildsuche

Beschrieben werden Techniken und Systeme zur zusammenstellungssensitiven Digitalbildsuche, die maschinelles Lernen einsetzen. Bei einem Beispiel setzt ein zur zusammenstellungssensitiven Bildsuche gegebenes System ein faltungstechnisches neuronales Two-Stream-Netzwerk (CNN) ein, um Merkmalseinbettungen aus Vordergrunddigitalbildern, die ein Vordergrundobjekt erfassen, und Hintergrunddigitalbildern, die eine Hintergrundszene erfassen, gemeinsam zu lernen. Zum Trainieren von Modellen der faltungstechnischen neuronalen Netzwerke werden Tripel von Trainingsdigitalbildern benutzt. Jedes Tripel kann ein Positivvordergrunddigitalbild und ein Positivhintergrunddigitalbild, die demselben Digitalbild entnommen sind, beinhalten. Das Tripel enthält zudem ein Negativvordergrund- oder Hintergrunddigitalbild, das zu dem Positivvordergrund- oder Hintergrunddigitalbild nicht ähnlich ist und das ebenfalls als Teil des Tripels beinhaltet ist.

Digitalbildvervollständigung unter Verwendung des Deep Learning

Beschrieben wird die Digitalbildvervollständigung unter Verwendung des Deep Learning. Zunächst wird ein Digitalbild mit wenigstens einem Loch empfangen. Dieses lochhaltige Digitalbild wird als Eingabe für einen Bildvervollständiger bereitgestellt, der mit einem Framework ausgebildet ist, das generative und diskriminative neuronale Netzwerke auf Grundlage der Lernarchitektur generativer adversativer Netzwerke kombiniert. Aus dem lochhaltigen Digitalbild generiert das generative neuronale Netzwerk ein gefülltes Digitalbild mit lochfüllendem Content anstelle der Löcher. Die diskriminativen neuronalen Netzwerke detektieren, ob das gefüllte Digitalbild und der lochfüllende Digitalcontent computerseitig generiertem Content entsprechen oder diesen beinhalten oder ob sie fotorealistisch sind. Das Generieren und Detektieren wird iterativ fortgesetzt, bis die diskriminativen neuronalen Netzwerke beim Detektieren von computerseitig generiertem Content für das gefüllte Digitalbild und lochfüllenden ...

Optimized scheduling method and system for cooperative harvest of agricultural machinery

The invention provides an optimized method and system for cooperative harvest of agricultural machinery. According to the method provided by the invention, firstly, an optimized objective function containing the number of the agricultural machinery as a variable is established based on an operation object, an optimization object, related information and constraint conditions; and then the optimized objective function is solved by adopting integer programming algorithm to obtain the optimized number of the agricultural machinery. Therefore, scientific scheduling of the agricultural machinery can be realized; optimization of indexes such as cost and fuel consumption in an operation process can be realized; working efficiency of the agricultural machinery, social benefits and economic benefits can be fully played; and mechanical level of the agricultural machinery can be further increased effectively.

Method and apparatus for determining a mobility of a mobile station in a wireless communication system

An infrastructure-based mobility determination module (MDM) receives multiple measurement reports from the MS and, for each received measurement report, stores values corresponding to values included in the report that are associated with measurements of parameters associated with received signals, wherein each stored value is stored in association with a wireless access node sourcing the corresponding signal. For each of one or more pairings of received measurement reports, the MDM then determines a signal parameter measurement change value corresponding to a change from the signal parametermeasurement values associated with a first measurement report of the pairing to the signal parameter measurement values associated with a second measurement report of the pairing. Based on the one ormore signal parameter measurement change values, the MDM determines an average signal parameter measurement change value and, based on the average signal parameter measurement change value, determines a ...

Automatically Segmenting Images Based On Natural Language Phrases

The invention is directed towards segmenting images based on natural language phrases. An image and an n-gram, including a sequence of tokens, are received. An encoding of image features and a sequence of token vectors are generated. A fully convolutional neural network identifies and encodes the image features. A word embedding model generates the token vectors. A recurrent neural network (RNN) iteratively updates a segmentation map based on combinations of the image feature encoding and the token vectors. The segmentation map identifies which pixels are included in an image region referenced by the n-gram. A segmented image is generated based on the segmentation map. The RNN may be a convolutional multimodal RNN. A separate RNN, such as a long short-term memory network, may iteratively update an encoding of semantic features based on the order of tokens. The first RNN may update the segmentation map based on the semantic feature encoding. 1. A computer-readable storage medium having instructions stored thereon for segmenting an image , which , when executed by a processor of a computing device cause the computing device to perform actions comprising:receiving a phrase that references a first region of the image, wherein the phrase includes a set of tokens;generating a plurality of token data elements based on the set of tokens, wherein each of the plurality of token data elements indicates a semantic feature of a corresponding token of the set of tokens;generating a plurality of iterative updates of a segmentation map of the image based on an order of the set of tokens, wherein each of a plurality of iterative updates of the segmentation map is based on the semantic feature indicated by the corresponding token data element; andsegmenting the first region of the image based on the iteratively updated segmentation map.2. The computer-readable storage medium of claim 1 , wherein the actions further comprise:generating an image map that represents a correspondence ...

Remote Radio Apparatus And Component Thereof

An example remote radio apparatus is provided, including a body, a mainboard, a mainboard heat sink, a maintenance cavity, an optical module, and an optical module heat sink. The maintenance cavity and the optical module heat sink are integrally connected, while the optical module is mounted on a bottom surface of the optical module heat sink. The maintenance cavity and the optical module heat sink are mounted on a side surface of the body, and the mainboard heat sink is mounted on and covers the mainboard. The mainboard heat sink and the mainboard are installed on a front surface of the body, and the mainboard heat sink and the optical module heat sink are spaced by a preset distance. The temperature of the optical module is controlled within a range required by a specification. 1. A remote radio apparatus (RRU) , comprising: a body , a mainboard , a mainboard heat sink , a maintenance cavity , an optical module , and an optical module heat sink , whereinthe maintenance cavity and the optical module heat sink are integrally connected;the optical module is mounted on a bottom surface of the optical module heat sink;the maintenance cavity and the optical module heat sink are mounted on a side surface of the body; andthe mainboard heat sink is mounted on and covers the mainboard, wherein the mainboard heat sink and the mainboard are mounted on a front surface of the body, and wherein the mainboard heat sink and the optical module heat sink are spaced by a preset distance.2. The RRU according to claim 1 , wherein support and connection between the mainboard heat sink and the optical module heat sink are implemented by using a plurality of support pieces claim 1 , and wherein the plurality of support pieces are configured to keep at least a portion of the preset distance between the mainboard heat sink and the optical module heat sink.3. The RRU according to claim 2 , wherein a first waterproof rubber strip is mounted at a gap within the preset distance between the ...

UTILIZING DEEP LEARNING TO RATE ATTRIBUTES OF DIGITAL IMAGES

Systems and methods are disclosed for estimating aesthetic quality of digital images using deep learning. In particular, the disclosed systems and methods describe training a neural network to generate an aesthetic quality score digital images. In particular, the neural network includes a training structure that compares relative rankings of pairs of training images to accurately predict a relative ranking of a digital image. Additionally, in training the neural network, an image rating system can utilize content-aware and user-aware sampling techniques to identify pairs of training images that have similar content and/or that have been rated by the same or different users. Using content-aware and user-aware sampling techniques, the neural network can be trained to accurately predict aesthetic quality ratings that reflect subjective opinions of most users as well as provide aesthetic scores for digital images that represent the wide spectrum of aesthetic preferences of various users. 120-. (canceled)21. A computer-implemented method of estimating aesthetic quality of digital images using deep learning , the method comprising:receiving a digital image;extracting a set of features from the digital image; andgenerating an attribute quality score for each of a plurality of attributes by processing the set of features using individual sets of fully-connected layers of a neural network; andgenerating an overall aesthetic quality score for the digital image by processing the set of features using a set of fully-connected layers of the neural network.22. The method as recited in claim 21 , further comprising generating an aesthetic quality score based on the attribute quality scores and the overall aesthetic quality score.23. The method as recited in claim 22 , wherein generating the aesthetic quality score comprises a numerical value indicating the overall aesthetic quality score and scores for each of the plurality of attributes.24. The method as recited in claim 22 , ...

Wheel rim rolling device and wheel rim rolling method

The invention belongs to a wheel rim rolling device in the technical field of wheel rim manufacturing, and further relates to a wheel rim rolling method. The height of a main roller hole pattern datum plane (6) of the main roller (1) is 5-15 mm higher than that of an upper roller face (10) of the lower conical roller, the upper end of the main roller (1) is provided with a double-inclination side wall (11) with the depth h, the height h of the side wall (11) is 0.5-0.8 time of the target thickness of the wheel rim, the angle beta 1 of the side wall (11) is 1-4 degrees, the angle beta 2 of the side wall (11) is 15-25 degrees, and the angle beta 2 of the side wall (11) is 1-4 degrees. The core roller (2) is connected with a driving part I capable of controlling the core roller (2) to move towards or away from the main roller (1), and the upper conical roller (4) is connected with a driving part II capable of controlling the upper conical roller (4) to move towards or away from the lower conical ...

RECURRENT NEURAL NETWORK ARCHITECTURES WHICH PROVIDE TEXT DESCRIBING IMAGES

Provided are systems and techniques that provide an output phrase describing an image. An example method includes creating, with a convolutional neural network, feature maps describing image features in locations in the image. The method also includes providing a skeletal phrase for the image by processing the feature maps with a first long short-term memory (LSTM) neural network trained based on a first set of ground truth phrases which exclude attribute words. Then, attribute words are provided by processing the skeletal phrase and the feature maps with a second LSTM neural network trained based on a second set of ground truth phrases including words for attributes. Then, the method combines the skeletal phrase and the attribute words to form the output phrase.

Method and apparatus for determining a mobility of a mobile station in a wireless communication system

An infrastructure-based mobility determination module (MDM) receives multiple measurement reports from the MS and, for each received measurement report, stores values corresponding to values included in the report that are associated with measurements of parameters associated with received signals, wherein each stored value is stored in association with a wireless access node sourcing the corresponding signal. For each of one or more pairings of received measurement reports, the MDM then determines a signal parameter measurement change value corresponding to a change from the signal parameter measurement values associated with a first measurement report of the pairing to the signal parameter measurement values associated with a second measurement report of the pairing. Based on the one or more signal parameter measurement change values, the MDM determines an average signal parameter measurement change value and, based on the average signal parameter measurement change value, determines ...

Digital Image Completion by Learning Generation and Patch Matching Jointly

Digital image completion by learning generation and patch matching jointly is described. Initially, a digital image having at least one hole is received. This holey digital image is provided as input to an image completer formed with a dual-stage framework that combines a coarse image neural network and an image refinement network. The coarse image neural network generates a coarse prediction of imagery for filling the holes of the holey digital image. The image refinement network receives the coarse prediction as input, refines the coarse prediction, and outputs a filled digital image having refined imagery that fills these holes. The image refinement network generates refined imagery using a patch matching technique, which includes leveraging information corresponding to patches of known pixels for filtering patches generated based on the coarse prediction. Based on this, the image completer outputs the filled digital image with the refined imagery. 1. A method comprising:generating, using a coarse image neural network, a coarse prediction of imagery to fill at least one hole of a digital image;generating, using an image refinement neural network, a refined fill by refining the coarse prediction, the coarse prediction refined by copying patches of pixels included in depicted imagery of the digital image based on a measure of similarity between the patches of pixels included in the depicted imagery of the digital image and patches of pixels of the coarse prediction; andreplacing the at least one hole of the digital image with the refined fill to form a filled digital image.2. A method as described in claim 1 , wherein the at least one hole comprises a set of pixels having values indicating an absence of the depicted imagery.3. A method as described in claim 1 , further comprising exposing the digital image to a dual-stage image completion framework that combines the coarse image neural network and the image refinement neural network.4. A method as described in ...

Agapanthus praecox gibberellin synthesis dioxygenase APGA20ox protein, and coding gene and probe thereof

The invention relates to an agapanthus praecox gibberellin synthesis dioxygenase APGA20ox protein, and a coding gene and a probe thereof. The protein is a protein consisting of the amino acid sequence shown in SEQ ID No.2. The invention also provides a corresponding nucleic acid sequence and a corresponding detection probe. According to steps, the 3' and 5' terminals of the APGA20ox gene are obtained by respective amplification through an RACE technology; the gene full-length sequence splicing and homology analysis are performed; and the expression difference condition of the APGA20ox gene is analyzed through spatio-temporal expression mode analysis and exogenous regulation substance treatment so as to verify the gene functions. To verify the functions of the APGA20ox, an agapanthus praecox embryonic callus is transformed by an RNAi technology, a dwarfing agapanthus praecox plant can be obtained by APGA20ox gene silencing, and therefore effects of the APGA20ox nucleic acid sequence on improvement ...

HARMONIZING COMPOSITE IMAGES USING DEEP LEARNING

Methods and systems are provided for generating harmonized images for input composite images. A neural network system can be trained, where the training includes training a neural network that generates harmonized images for input composite images. This training is performed based on a comparison of a training harmonized image and a reference image, where the reference image is modified to generate a training input composite image used to generate the training harmonized image. In addition, a mask of a region can be input to limit the area of the input image that is to be modified. Such a trained neural network system can be used to input a composite image and mask pair for which the trained system will output a harmonized image.

Cascaded Object Detection

Cascaded object detection techniques are described. In one or more implementations, cascaded coarse-to-dense object detection techniques are utilized to detect objects in images. In a first stage, coarse features are extracted from an image, and non-object regions are rejected. Then, in one or more subsequent stages, dense features are extracted from the remaining non-rejected regions of the image to detect one or more objects in the image. 1. A computer-implemented method comprising:receiving an image; extracting features from the image;', 'identifying, based on the extracted features, non-object regions of the image which do not include an object;', 'rejecting the non-object regions of the image;, 'for a first stage extracting additional features from non-rejected regions of the image;', 'identifying, based on the additional extracted features, additional non-object regions of the image;', 'rejecting the additional non-object regions of the image;, 'for one or more subsequent stagesfor a final stage of the one or more subsequent stages, detecting one or more objects in the image based on features extracted in the final stage.2. The computer-implemented method of claim 1 , wherein the identifying the non-object regions of the image further comprises:computing a confidence score for each region that indicates a confidence that the region includes the object;comparing the confidence score for each region to a low threshold; andidentifying regions with confidence scores that are less than the low threshold as the non-object regions of the image.3. The computer-implemented method of claim 1 , further comprising claim 1 , for the first stage and the one or more subsequent stages claim 1 , identifying claim 1 , based on the extracted features claim 1 , object regions of the image which include the object.4. The computer-implemented method of claim 3 , wherein the identifying the object regions of the image further comprises:computing a confidence score for each region ...

UTILIZING DEEP LEARNING FOR BOUNDARY-AWARE IMAGE SEGMENTATION

Systems and methods are disclosed for segmenting a digital image to identify an object portrayed in the digital image from background pixels in the digital image. In particular, in one or more embodiments, the disclosed systems and methods use a first neural network and a second neural network to generate image information used to generate a segmentation mask that corresponds to the object portrayed in the digital image. Specifically, in one or more embodiments, the disclosed systems and methods optimize a fit between a mask boundary of the segmentation mask to edges of the object portrayed in the digital image to accurately segment the object within the digital image. 1. In a digital medium environment for editing digital visual media , a method of using deep learning to segment objects from digital visual media , the method comprising:generating, by at least one processor with a first neural network, a probability map for an input image, wherein the probability map indicates object pixels predicted to correspond to an object portrayed in the input image;generating, by the at least one processor with a second neural network, a boundary map for the input image, wherein the boundary map indicates edge pixels predicted to correspond to edges of the object portrayed in the input image;based on the probability map and the boundary map, generating, by the at least one processor, a segmentation mask for the object by optimizing a fit between a mask boundary of the object and the edges of the object; andbased on the segmentation mask, identifying, by the at least one processor, a set of pixels corresponding to the object portrayed in the input image.2. The method of claim 1 , wherein optimizing the fit between the mask boundary of the object and the edges of the object comprises an iterative optimization process using a combination of color modeling and boundary modeling.3. The method of claim 1 , further comprising generating a boundary refinement map to determine ...

Utilizing deep learning to rate attributes of digital images

Systems and methods are disclosed for estimating aesthetic quality of digital images using deep learning. In particular, the disclosed systems and methods describe training a neural network to generate an aesthetic quality score digital images. In particular, the neural network includes a training structure that compares relative rankings of pairs of training images to accurately predict a relative ranking of a digital image. Additionally, in training the neural network, an image rating system can utilize content-aware and user-aware sampling techniques to identify pairs of training images that have similar content and/or that have been rated by the same or different users. Using content-aware and user-aware sampling techniques, the neural network can be trained to accurately predict aesthetic quality ratings that reflect subjective opinions of most users as well as provide aesthetic scores for digital images that represent the wide spectrum of aesthetic preferences of various users.

Depth-of-field blur effects generating techniques

Techniques of generating depth-of-field blur effects on digital images by digital effect generation system of a computing device are described. The digital effect generation system is configured to generate depth-of-field blur effects on objects based on focal depth value that defines a depth plane in the digital image and a aperture value that defines an intensity of blur effect applied to the digital image. The digital effect generation system is also configured to improve the accuracy with which depth-of-field blur effects are generated by performing up-sampling operations and implementing a unique focal loss algorithm that minimizes the focal loss within digital images effectively.

Method for inducing adventitious bud of Schisandga Chinensis baill

The invention relates to an adventitious bud induction method for group culturing and rapid propagating Schisandra Chinensis in the filed of agricultural technology, which adopts the top and the middle part stem segments that are provided with an axillary bud of the Schisandra Chinensis shoot as an explant for tissue culture, uses a saturated sodium hypochlorite discontinuous sterilization for explant disinfection and inoculates the explant in an induction medium under aseptic operation and is cultured under the conditions of a temperature of 20DEG C to 25DEG C, illuminating for 12h/d with an illumination intensity of 800-1200lx. The induction medium comprises the following components: each liter liquid MS basic medium with 2.0mg of 6-BA, 0.05mg of Alpha-NAA, 0.06mg of zeatin, 30g of cane sugar and 8g of agar. After 20 to 30 days, the stem segment of Schisandra Chinensis shoots the axillary bud. The adventitious bud is cut from the explant a few days later when the axillary bud is 2cm, ...

K-nearest neighbor re-ranking

Methods, apparatus, and computer-readable storage media for k-NN re-ranking. Based on retrieved images and localized objects, a k-NN re-ranking method may use the k-nearest neighbors of a query to refine query results. Given the top k retrieved images and their localized objects, each k-NN object may be used as a query to perform a search. A database image may have different ranks when using those k-nearest neighbors as queries. Accordingly, a new score for each database image may be collaboratively determined by those ranks, and re-ranking may be performed using the new scores to improve the search results. The k-NN re-ranking technique may be performed two or more times, each time on a new set of k-nearest neighbors, to further refine the search results.

Flat TV set

Inside the cylinder with retaining ring at the projection exit of common projection TV set, are set a projection lens and a plane mirror, and the projection exit is connected to display screen comprising optical-fibre panel, optical-fibre image-transmitting bundle and casing. High-brightness image is projected onto the small end face of optical-fibre image-transmitting bundle and transmitted to large end face. The optical-fibre panel has same material as optical fibre and inclined end face so that it enlarges the image. The flat TV set has large display area, thin display screen, high brightness and low production cost.

FRAME SELECTION BASED ON A TRAINED NEURAL NETWORK

Various embodiments describe frame selection based on training and using a neural network. In an example, the neural network is a convolutional neural network trained with training pairs. Each training pair includes two training frames from a frame collection. The loss function relies on the estimated quality difference between the two training frames. Further, the definition of the loss function varies based on the actual quality difference between these two frames. In a further example, the neural network is trained by incorporating facial heatmaps generated from the training frames and facial quality scores of faces detected in the training frames. In addition, the training involves using a feature mean that represents an average of the features of the training frames belonging to the same frame collection. Once the neural network is trained, a frame collection is input thereto and a frame is selected based on generated quality scores. 1. A computer-implemented method of using a neural network to select a frame from a collection of frames , the computer-implemented method comprising:accessing, by a computer system, training data that comprises training frames and training labels, the training frames associated with a same scene, each training label associated with a training frame of the training frames and indicating a quality of the training frame; generating a training pair that comprises a first training frame and a second training frame from the training frames, the training pair generated based on the first training frame having a higher quality than the second training frame according to the training labels,', 'generating a first quality difference between the first training frame and the second training frame in the training pair based on a comparison of a first training label and a second training label, the first training label associated with the first training frame, and the second training label associated with the second training frame,', 'inputting ...

Automatically selecting example stylized images for image stylization operations based on semantic content

Systems and methods are provided for content-based selection of style examples used in image stylization operations. For example, training images can be used to identify example stylized images that will generate high-quality stylized images when stylizing input images having certain types of semantic content. In one example, a processing device determines which example stylized images are more suitable for use with certain types of semantic content represented by training images. In response to receiving or otherwise accessing an input image, the processing device analyzes the semantic content of the input image, matches the input image to at least one training image with similar semantic content, and selects at least one example stylized image that has been previously matched to one or more training images having that type of semantic content. The processing device modifies color or contrast information for the input image using the selected example stylized image.

Landmark localization via visual search

One exemplary embodiment involves identifying feature matches between each of a plurality of object images and a test image, each of the feature matches between a feature of a respective object image and a matching feature of the test image, wherein there is a spatial relationship between each respective object image feature and a first landmark of the object image, the first landmark at a known location in the object image. The embodiment additionally involves estimating a plurality of locations for a second landmark for the test image, the estimated locations based at least in part on the feature matches and the spatial relationships, and estimating a final location for the second landmark from the plurality of locations for the second landmark for the test image.

Method for regulating and controlling performances of bainite non-tempered steel for fasteners

The invention discloses a method for regulating and controlling performances of bainite non-tempered steel for fasteners, belonging to the technical field of controlled rolling and controlled cooling of steel. The method is characterized by comprising the following steps: preparing the bainite non-tempered steel for fasteners by adopting high-speed wire rolling equipment, wherein the conventional rolling temperature is adopted at the crude, medium and pre-finish rolling stages of high-speed wire rolling, and the low temperature rolling technology is adopted at the finish rolling stage; controlling the austenite deformation temperature at 750-900 DEG C, wherein the deformation is not less than 40%; and controlling cooling after deformation, wherein controlled cooling is divided into single cooling and subsection cooling, the single cooling speed is 3-5 DEG C/s, rapid cooling at a speed not less than 5 DEG C/s is adopted before bainite phase transformation in subsection cooling and then slow ...

Deep salient content neural networks for efficient digital object segmentation

Systems, methods, and non-transitory computer-readable media are disclosed for segmenting objects in digital visual media utilizing one or more salient content neural networks. In particular, in one or more embodiments, the disclosed systems and methods train one or more salient content neural networks to efficiently identify foreground pixels in digital visual media. Moreover, in one or more embodiments, the disclosed systems and methods provide a trained salient content neural network to a mobile device, allowing the mobile device to directly select salient objects in digital visual media utilizing a trained neural network. Furthermore, in one or more embodiments, the disclosed systems and methods train and provide multiple salient content neural networks, such that mobile devices can identify objects in real-time digital visual media feeds (utilizing a first salient content neural network) and identify objects in static digital images (utilizing a second salient content neural network ...

Self-healing method and device for signal acquisition abnormity of excitation control acquisition board

The invention specifically relates to an excitation system acquisition board signal acquisition abnormity self-healing method comprising the following steps: an AD acquisition circuit is constructed on an excitation control acquisition board, the AD acquisition circuit is composed of a plurality of AD chips, one AD chip is used as a master chip for data reading, and the other AD chips are used as slave chips for data backup; a plurality of AD chips are synchronously sampled, so that the problems of discontinuous sampling and reduced sampling precision are avoided; when one AD chip is abnormal in the sampling process, switching to use the data of the other AD chips which are normally converted; and self-healing is carried out on abnormities occurring in the sampling process of the AD chip. According to the self-healing method for the signal acquisition abnormity of the excitation system acquisition board, self-healing is carried out on the signal acquisition abnormity of the excitation control ...

Image cropping suggestion using multiple saliency maps

Image cropping suggestion using multiple saliency maps is described. In one or more implementations, component scores, indicative of visual characteristics established for visually-pleasing croppings, are computed for candidate image croppings using multiple different saliency maps. The visual characteristics on which a candidate image cropping is scored may be indicative of its composition quality, an extent to which it preserves content appearing in the scene, and a simplicity of its boundary. Based on the component scores, the croppings may be ranked with regard to each of the visual characteristics. The rankings may be used to cluster the candidate croppings into groups of similar croppings, such that croppings in a group are different by less than a threshold amount and croppings in different groups are different by at least the threshold amount. Based on the clustering, croppings may then be chosen, e.g., to present them to a user for selection.

Attribute recognition via visual search

One exemplary embodiment involves identifying feature matches between each of a plurality of object images and a test image, each feature matches between a feature of a respective object image and a matching feature of the test image, wherein there is a spatial relationship between each respective object image feature and a test image feature, and wherein the object depicted in the test image comprises a plurality of attributes. Additionally, the embodiment involves estimating, for each attribute in the test image, an attribute value based at least in part on information stored in a metadata associated with each of the object images.

Metal 3D printing welding wire cleaning device and cleaning method

The invention relates to a metal 3D printing welding wire cleaning device which comprises a cleaning unit, a drying unit and a take-up machine which are arranged in sequence, the cleaning unit comprises an alkaline cleaning tank, a first hot water cleaning tank, an acid pickling tank and at least two second hot water cleaning tanks which are arranged in sequence, and the alkaline cleaning tank and the acid pickling tank are each provided with an ultrasonic generation assembly. The two ends of the alkaline washing tank, the two ends of the first hot water cleaning tank, the two ends of the pickling tank, the two ends of the second hot water cleaning tank and the two ends of the drying unit are each provided with a wire guide wheel, and during running, the metal 3D printing welding wire passes through the alkaline washing tank, the first hot water cleaning tank, the pickling tank and the second hot water cleaning tank along the wire guide wheels to be treated; and the air passes through the ...

Semantic Class Localization Digital Environment

Semantic segmentation techniques and systems are described that overcome the challenges of limited availability of training data to describe the potentially millions of tags that may be used to describe semantic classes in digital images. In one example, the techniques are configured to train neural networks to leverage different types of training datasets using sequential neural networks and use of vector representations to represent the different semantic classes. 1. In a digital medium semantic class localization environment , a method implemented by a least one computing device , the method comprising:converting, by the at least one computing device, a tag into a vector representation, the tag defining a semantic class to be located in a digital image;generating, by the at least one computing device, an attention map by an embedding neural network based on the digital image and the vector representation, the attention map defining a location in the digital image that corresponds to the semantic class, the embedding neural network trained using image-level tags of respective semantic classes;refining, by the at least one computing device, the location of the semantic class in the attention map by a refinement neural network, the refinement neural network trained using localized tags of respective semantic classes; andindicating, by the at least one computing device, the refined location of the semantic class in the digital image using the refined attention map.2. The method as described in claim 1 , wherein the converting of the vector representation uses an embedding neural network as part of machine learning.3. The method as described in claim 1 , wherein the image-level tags indicate respective semantic classes that are associated with respective digital images as a whole that are used to train the embedding neural network.4. The method as described in claim 1 , wherein the image-level tags are not localized to respective portions of digital images that are ...

Recurrent neural network architectures which provide text describing images

Provided are systems and techniques that provide an output phrase describing an image. An example method includes creating, with a convolutional neural network, feature maps describing image features in locations in the image. The method also includes providing a skeletal phrase for the image by processing the feature maps with a first long short-term memory (LSTM) neural network trained based on a first set of ground truth phrases which exclude attribute words. Then, attribute words are provided by processing the skeletal phrase and the feature maps with a second LSTM neural network trained based on a second set of ground truth phrases including words for attributes. Then, the method combines the skeletal phrase and the attribute words to form the output phrase.

Object detection using cascaded convolutional neural networks

Different candidate windows in an image are identified, such as by sliding a rectangular or other geometric shape of different sizes over an image to identify portions of the image (groups of pixels in the image). The candidate windows are analyzed by a set of convolutional neural networks, which are cascaded so that the input of one convolutional neural network layer is based on the input of another convolutional neural network layer. Each convolutional neural network layer drops or rejects one or more candidate windows that the convolutional neural network layer determines does not include an object (e.g., a face). The candidate windows that are identified as including an object (e.g., a face) are analyzed by another one of the convolutional neural network layers. The candidate windows identified by the last of the convolutional neural network layers are the indications of the objects (e.g., faces) in the image.

METHOD AND APPARATUS FOR DETERMINING A MOBILITY OF A MOBILE STATION IN A WIRELESS COMMUNICATION SYSTEM

An infrastructure-based mobililty determination module (MDM) receives multiple measurement reports from the MS and, for each received measurement report, stores values corresponding to values included in the report that are associated with measurements of parameters associated with received signals, wherein each stored value is stored in association with a wireless access node sourcing the corresponding signal. For each of one or more pairings of received measurement reports, the MDM then determines a signal parameter measurement change value corresponding to a change from the signal parameter measurement values associated with a first measurement report of the pairing to the signal parameter measurement values associated with a second measurement report of the pairing. Based on the one or more signal parameter measurement change values, the MDM determines an average signal parameter measurement change value and, based on the average signal parameter measurement change value, determines ...

Remote radio apparatus and component thereof

An example remote radio apparatus is provided, including a body, a mainboard, a mainboard heat sink, a maintenance cavity, an optical module, and an optical module heat sink. The maintenance cavity and the optical module heat sink are integrally connected, while the optical module is mounted on a bottom surface of the optical module heat sink. The maintenance cavity and the optical module heat sink are mounted on a side surface of the body, and the mainboard heat sink is mounted on and covers the mainboard. The mainboard heat sink and the mainboard are installed on a front surface of the body, and the mainboard heat sink and the optical module heat sink are spaced by a preset distance. The temperature of the optical module is controlled within a range required by a specification.

Vorhersagen von Patchverschiebungsabbildungen unter Verwendung eines Neuronalnetzwerkes

Beschrieben wird das Vorhersagen von Patchverschiebungsabbildungen unter Verwendung eines Neuronalnetzwerkes. Zunächst wird ein Digitalbild, an dem ein Bildbearbeitungsvorgang durchgeführt werden soll, als Eingabe für einen Patchabgleicher, der ein Offsetvorhersage-Neuronalnetzwerk aufweist, bereitgestellt. Aus diesem Bild generiert das Offsetvorhersage-Neuronalnetzwerk auf Grundlage des Bildbearbeitungsvorganges, für den das Netzwerk trainiert wird, eine Offsetvorhersage, die als Verschiebungsabbildung ausgebildet ist, die Offsetvektoren aufweist, die eine Verschiebung von Pixeln des Digitalbildes an verschiedene bzw. andere Orte zum Durchführen des Bildverarbeitungsvorganges darstellen. Pixelwerte des Digitalbildes werden auf die Bildpixel, die von dem Vorgang betroffen sind, kopiert durch: Bestimmen der Vektorpixel, die den von dem Bildbearbeitungsvorgang betroffenen Bildpixeln entsprechen, und Abbilden der Pixelwerte der von den bestimmten Offsetvektoren dargestellten Bildpixel auf ...

PERFORMING ATTRIBUTE-AWARE BASED TASKS VIA AN ATTENTION-CONTROLLED NEURAL NETWORK

This disclosure covers methods, non-transitory computer readable media, and systems that learn attribute attention projections for attributes of digital images and parameters for an attention controlled neural network. By iteratively generating and comparing attribute-modulated-feature vectors from digital images, the methods, non-transitory computer readable media, and systems update attribute attention projections and parameters indicating either one (or both) of a correlation between some attributes of digital images and a discorrelation between other attributes of digital images. In certain embodiments, the methods, non-transitory computer readable media, and systems use the attribute attention projections in an attention controlled neural network as part of performing one or more tasks. 1. A system for training attention controlled neural networks to generate attribute-modulated-feature vectors using attribute attention projections comprising:at least one processor; generate at least one attribute attention projection for at least one attribute category of training images of the plurality of training images;', 'utilize the at least one attribute attention projection to generate at least one attribute-modulated-feature vector for at least one training image of the training images by inserting the at least one attribute attention projection between at least one set of layers of the attention controlled neural network; and', 'jointly learn at least one updated attribute attention projection and updated parameters of the attention controlled neural network by minimizing a loss from a loss function based on the at least one attribute-modulated-feature vector., 'at least one non-transitory computer memory comprising an attention controlled neural network, a plurality of training images, and instructions that, when executed by at least one processor, cause the system to2. The system of claim 1 , further comprising instructions that claim 1 , when executed by the at ...

Object detection with boosted exemplars

In techniques for object detection with boosted exemplars, weak classifiers of a real-adaboost technique can be learned as exemplars that are collected from example images. The exemplars are examples of an object that is detectable in image patches of an image, such as faces that are detectable in images. The weak classifiers of the real-adaboost technique can be applied to the image patches of the image, and a confidence score is determined for each of the weak classifiers as applied to an image patch of the image. The confidence score of a weak classifier is an indication of whether the object is detected in the image patch of the image based on the weak classifier. All of the confidence scores of the weak classifiers can then be summed to generate an overall object detection score that indicates whether the image patch of the image includes the object.

Remote radio apparatus and component thereof

An example remote radio apparatus is provided, including a body, a mainboard, a mainboard heat sink, a maintenance cavity, an optical module, and an optical module heat sink. The maintenance cavity and the optical module heat sink are integrally connected, while the optical module is mounted on a bottom surface of the optical module heat sink. The maintenance cavity and the optical module heat sink are mounted on a side surface of the body, and the mainboard heat sink is mounted on and covers the mainboard. The mainboard heat sink and the mainboard are installed on a front surface of the body, and the mainboard heat sink and the optical module heat sink are spaced by a preset distance. The temperature of the optical module is controlled within a range required by a specification.

Tiefe neurale Netzwerke für hervorstechenden Inhalt für eine effiziente Segmentierung eines digitalen Objekts

Systeme, Verfahren und nicht-flüchtige computerlesbare Medien werden für ein Segmentieren von Objekten in digitalen visuellen Medien geoffenbart, welche ein oder mehrere neurale(s) Netzwerk(e) für hervorstechenden Inhalt verwenden. Insbesondere trainieren in einer oder mehreren Ausführungsform(en) die geoffenbarten Systeme und Verfahren ein oder mehrere neurale(s) Netzwerk(e) für hervorstechenden Inhalt, um effizient Vordergrundpixel in digitalen visuellen Medien zu identifizieren. Darüber hinaus stellen in einer oder mehreren Ausführungsform(en) die geoffenbarten Systeme und Verfahren ein trainiertes neurales Netzwerk für hervorstechenden Inhalt an einer mobilen Vorrichtung zur Verfügung, welche erlauben, dass die mobile Vorrichtung direkt hervorstechende Objekte in digitalen visuellen Medien unter Verwendung eines trainierten neuralen Netzwerks auswählt. Darüber hinaus trainieren in einer oder mehreren Ausführungsform(en) die geoffenbarten Systeme und Verfahren mehrfache neurale Netzwerke ...

Method and Apparatus for Determining Motion

An apparatus comprising a processor and a memory that cause the apparatus to perform receiving a video indicating a motion, generating a set of scalar representations of movement based, at least in part, on at least part of the video, and identifying at least one predetermined motion that correlates to the set of scalar representations of movement is disclosed. 1. An apparatus , comprising:a processor;memory including computer program code, the memory and the computer program code configured to, working with the processor, cause the apparatus to perform at least the following:receiving a video indicating a motion;generating a set of scalar representations of movement based, at least in part, on at least part of the video; andidentifying at least one predetermined motion that correlates to the set of scalar representations of movement.2. The apparatus of claim 1 , wherein generating the set of scalar representations of movement is performed absent performance of a tracking calculation.3. The apparatus of claim 1 , wherein generating the set of scalar representations of movement is performed absent performance of a segmentation calculation.4. The apparatus of claim 1 , wherein generating the set of scalar representations of movement comprises selecting a set of frame images from at least part of the video claim 1 , wherein the set of scalar representations of movement are based claim 1 , at least in part claim 1 , on the set of frame images.5. The apparatus of claim 4 , wherein generating the set of scalar representations of movement comprises calculating an estimated optical flow based on the set of frame images claim 4 , wherein the set of scalar representations of movement are based claim 4 , at least in part claim 4 , on the estimated optical flow.6. The apparatus of claim 4 , wherein the memory and the computer program code are further configured to claim 4 , working with the processor claim 4 , cause the apparatus to further perform at least determining a video ...

REAL-TIME LOCATION ESTIMATION OF A MOBILE STATION IN A WIRELESS COMMUNICATION SYSTEM

An apparatus and method is provided for real-time location estimation of a mobile station (MS) on a map of roads using non-Global Positioning Satellite location data () of the MS and includes a step of determining () a mobility of the MS. A next step () includes identifying a region that covers all location data. A next step () includes dividing the region into smaller blocks, where each block constitutes a Hidden Markov Model state. A next step () includes determining a distance between blocks, to be used in a varied continuous probability distribution to determining a state transition probability of each block to represent a likelihood of the MS moving to any one block. A next step () includes using a univariate continuous distribution as a function of a distance between each block and a raw location data. A next step () includes finding a most likely state/block sequence of the MS motion using the HMM state transition probability and the univariate continuous distribution as the location of the MS. 1. A method for real-time location estimation of a mobile station in a wireless communication system on a geographic map of roads using non-Global Positioning Satellite (non-GPS) raw location data of the mobile station and depending on a mobility status of the mobile station , the method comprising the steps of: identifying a region on the map with a minimum area that covers all raw location data;', 'dividing all the areas bordered by roads within the region into smaller blocks, wherein each block constitute a Hidden Markov Model (HMM) state;', 'for each block, determining a number of blocks between that block and all other blocks, and using the number of blocks in a varied continuous probability distribution to determining a HMM state transition probability of each block to represent a likelihood of the mobile station moving from one block to another;', 'using a univariate continuous distribution as a function of a distance between each block and a raw location data; ...

OBJECT RETRIEVAL AND LOCALIZATION USING A SPATIALLY-CONSTRAINED SIMILARITY MODEL

Methods, apparatus, and computer-readable storage media for object retrieval and localization that employ a spatially-constrained similarity model. A spatially-constrained similarity measure may be evaluated by a voting-based scoring technique. Object retrieval and localization may thus be achieved without post-processing. The spatially-constrained similarity measure may handle object rotation, scaling and view point change. The similarity measure can be efficiently calculated by the voting-based method and integrated with inverted files. The voting-based scoring technique may simultaneously retrieve and localize a query object in a collection of images such as an image database. The object retrieval and localization technique may, for example, be implemented with a k-nearest neighbor (k-NN) re-ranking method in or as a retrieval method, system or module. The k-NN re-ranking method may be applied to improve query results of the object retrieval and localization technique. 1. A method , comprising: obtaining a query object for a query image, wherein the query object is represented by a bounding box within the query image and indications of a plurality of features of the query image located within the bounding box;', 'generating a plurality of geometric transforms of the query object;', 'calculating a similarity score for each of the plurality of transforms with respect to a target image according to a spatially-constrained similarity measure that accounts for rotation, scale, and translation; and', 'selecting the transform with a highest similarity score, wherein the transform with the highest similarity score indicates a localized object in the target image that best matches the query object., 'performing, by one or more computing devices2. The method as recited in claim 1 , wherein said generating a plurality of geometric transforms of the query object comprises rotating and scaling the query object according to each combination of a plurality of rotation angles ...

K-NEAREST NEIGHBOR RE-RANKING

Methods, apparatus, and computer-readable storage media for k-NN re-ranking. Based on retrieved images and localized objects, a k-NN re-ranking method may use the k-nearest neighbors of a query to refine query results. Given the top k retrieved images and their localized objects, each k-NN object may be used as a query to perform a search. A database image may have different ranks when using those k-nearest neighbors as queries. Accordingly, a new score for each database image may be collaboratively determined by those ranks, and re-ranking may be performed using the new scores to improve the search results. The k-NN re-ranking technique may be performed two or more times, each time on a new set of k-nearest neighbors, to further refine the search results. 1. A method for object retrieval and localization , comprising: obtaining an initial ranking of a collection with regard to a query object;', 'generating a ranking of the collection with regard to each of one or more nearest neighbors to the query object as indicated by the initial ranking of the collection; and', 'generating a new ranking of the collection with regard to the query object according to the initial ranking and the generated rankings with regard to each of the one or more nearest neighbors., 'performing, by one or more computing devices2. The method as recited in claim 1 , wherein the collection is a plurality of images claim 1 , and wherein the initial ranking is a ranking of similarity of each of the plurality of images to a query image that includes the query object.3. The method as recited in claim 1 , wherein said generating a ranking of the collection with regard to each of one or more nearest neighbors to the query object as indicated by the initial ranking of the collection comprises:for each of the one or more nearest neighbors, searching the collection according to a localized object of the respective nearest neighbor to determine similarity scores for the collection; andranking the ...

GENERATING IMAGE FEATURES BASED ON ROBUST FEATURE-LEARNING

Techniques for increasing robustness of a convolutional neural network based on training that uses multiple datasets and multiple tasks are described. For example, a computer system trains the convolutional neural network across multiple datasets and multiple tasks. The convolutional neural network is configured for learning features from images and accordingly generating feature vectors. By using multiple datasets and multiple tasks, the robustness of the convolutional neural network is increased. A feature vector of an image is used to apply an image-related operation to the image. For example, the image is classified, indexed, or objects in the image are tagged based on the feature vector. Because the robustness is increased, the accuracy of the generating feature vectors is also increased. Hence, the overall quality of an image service is enhanced, where the image service relies on the image-related operation. 1. A computer-implemented method associated with using a convolutional neural network , the method comprising:accessing, by a computer system, a first training dataset comprising first image data, the first training dataset associated with a first task and a first label applicable to the first image data; minimizing a first loss function for the first training dataset based on the first task and a second loss function for the second training dataset based on the second task, and', 'updating parameters of the convolutional neural network based on the minimizing of the first loss function and the second loss function., 'training, by the computer system, the convolutional neural network by at least, 'accessing, by the computer system, a second training dataset comprising second training data, the second training dataset associated with a second task and a second label; and'}2. The computer-implemented method of claim 1 , further comprising:inputting, by the computer system, image data of an image to the convolutional neural network; andgenerating, by the ...

Feature Interpolation

Feature interpolation techniques are described. In a training stage, features are extracted from a collection of training images and quantized into visual words. Spatial configurations of the visual words in the training images are determined and stored in a spatial configuration database. In an object detection stage, a portion of features of an image are extracted from the image and quantized into visual words. Then, a remaining portion of the features of the image are interpolated using the visual words and the spatial configurations of visual words stored in the spatial configuration database.

Accelerating Object Detection

Accelerating object detection techniques are described. In one or more implementations, adaptive sampling techniques are used to extract features from an image. Coarse features are extracted from the image and used to generate an object probability map. Then, dense features are extracted from high-probability object regions of the image identified in the object probability map to enable detection of an object in the image. In one or more implementations, cascade object detection techniques are used to detect an object in an image. In a first stage, exemplars in a first subset of exemplars are applied to features extracted from the multiple regions of the image to detect object candidate regions. Then, in one or more validation stages, the object candidate regions are validated by applying exemplars from the first subset of exemplars and one or more additional subsets of exemplars. 1. A computer-implemented method comprising:receiving an image;extracting coarse features from the image;generating an object probability map based on the coarse features extracted from the image, the object probability map indicating high-probability object regions in the image that are likely to contain an object; andextracting dense features from the high-probability object regions of the image identified in the object probability map to enable detection of one or more objects in the image.2. The computer-implemented method of claim 1 , wherein the object probability map further indicates low-probability object regions in the image that are not likely to contain an object claim 1 , and wherein dense features are not extracted from the low-probability object regions of the image.3. The computer-implemented method of claim 1 , further comprising detecting one or more objects in the image based on the dense features extracted from the image.4. The computer-implemented method of claim 1 , wherein the generating the object probability map comprises:quantizing the extracted features into ...

Shortlist computation for searching high-dimensional spaces

Techniques are disclosed for indexing and searching high-dimensional data using inverted file structures and product quantization encoding. An image descriptor is quantized using a form of product quantization to determine which of several inverted lists the image descriptor is to be stored. The image descriptor is appended to the corresponding inverted list with a compact coding using a product quantization encoding scheme. When processing a query, a shortlist is computed that includes a set of candidate search results. The shortlist is based on the orthogonality between two random vectors in high-dimensional spaces. The inverted lists are traversed in the order of the distance between the query and the centroid of a coarse quantizer corresponding to each inverted list. The shortlist is ranked according to the distance estimated by a form of product quantization, and the top images referred to by the ranked shortlist are reported as the search results.

PLANAR REGION GUIDED 3D GEOMETRY ESTIMATION FROM A SINGLE IMAGE

Techniques for planar region-guided estimates of 3D geometry of objects depicted in a single 2D image. The techniques estimate regions of an image that are part of planar regions (i.e., flat surfaces) and use those planar region estimates to estimate the 3D geometry of the objects in the image. The planar regions and resulting 3D geometry are estimated using only a single 2D image of the objects. Training data from images of other objects is used to train a CNN with a model that is then used to make planar region estimates using a single 2D image. The planar region estimates, in one example, are based on estimates of planarity (surface plane information) and estimates of edges (depth discontinuities and edges between surface planes) that are estimated using models trained using images of other scenes. 1. A method , performed by a computing device , for enhancing an image based on planar-region-guided estimates of 3D geometry of objects depicted in the image , the method comprising:determining planarity and edge strength of pixels of the image, wherein the determining of planarity and edge strength of the pixels of the image is based on the image of the objects and is not based on additional images of the objects;determining whether pixels of the image are within common planar regions based on the determining of planarity and edge strength of the pixels of the image;determining 3D geometry values of pixels in the common planar regions based on a planar region constraint that requires a relationship between the 3D geometry values of pixels within common planar regions; andenhancing the image by using the 3D geometry values of the pixels to provide the 3D geometry of the objects in the image.2. The method of claim 1 , wherein determining the 3D geometry values of the pixels in common planar regions comprises selecting normals of the pixels of the image based on the planar region constraint requiring similar normals of pixels in common planar regions.3. The method of ...

HIGH DYNAMIC RANGE ILLUMINATION ESTIMATION

Systems and techniques for estimating illumination from a single image are provided. An example system may include a neural network. The neural network may include an encoder that is configured to encode an input image into an intermediate representation. The neural network may also include an intensity decoder that is configured to decode the intermediate representation into an output light intensity map. An example intensity decoder is generated by a multi-phase training process that includes a first phase to train a light mask decoder using a set of low dynamic range images and a second phase to adjust parameters of the light mask decoder using a set of high dynamic range image to generate the intensity decoder. 1. A system for estimating illumination , the system comprising:at least one memory including instructions; and an encoder that is configured to encode an input image into an intermediate representation; and', 'an intensity decoder that is configured to decode the intermediate representation into an output light intensity map., 'at least one processor that is operably coupled to the at least one memory and that is arranged and configured to execute instructions that, when executed, cause the at least one processor to implement a neural network trained based on low dynamic range training images and refined based on high dynamic range training images, the neural network comprising2. The system of claim 1 , wherein a number of low dynamic range training images used to train the neural network is larger than a number of high dynamic range training images used to refine the neural network.3. The system of claim 1 , wherein the neural network further comprises an environment map decoder configured to decode an output environment map from the intermediate representation.4. The system of claim 1 , wherein the low dynamic range training images includes low dynamic range training panoramic images and the neural network trained based on low dynamic range training ...

Cropping Boundary Simplicity

Cropping boundary simplicity techniques are described. In one or more implementations, multiple candidate cropping s of a scene are generated. For each of the candidate croppings, a score is calculated that is indicative of a boundary simplicity for the candidate cropping. To calculate the boundary simplicity, complexity of the scene along a boundary of a respective candidate cropping is measured. The complexity is measured, for instance, using an average gradient, an image edge map, or entropy along the boundary. Values indicative of the complexity may be derived from the measuring. The candidate croppings may then be ranked according to those values. Based on the scores calculated to indicate the boundary simplicity, one or more of the candidate croppings may be chosen e.g., to present the chosen croppings to a user for selection. 1. A method implemented by a computing device for suggesting croppings of an image , the method comprising:generating a blurred version of the image;extracting a gradient map of the blurred version of the image;generating a plurality of candidate croppings of the image;calculating, for each of the candidate cropping s, an average gradient value along a boundary of the candidate cropping from the gradient map;ranking the candidate croppings according to the average gradient values; andchoosing at least one cropping from the candidate croppings based on the ranking.2. A method as described in claim 1 , wherein the candidate croppings are rectangularly shaped and calculating the average gradient value along a rectangular boundary includes calculating an average gradient for each of an upper side claim 1 , bottom side claim 1 , left side claim 1 , and right side of the rectangular boundary.3. A method as described in claim 1 , wherein the average gradient value is indicative of the complexity of the boundary of the candidate cropping.4. A method as described in claim 1 , wherein the ranking ranks the candidate croppings that have average ...

Image Cropping Suggestion Using Multiple Saliency Maps

Image cropping suggestion using multiple saliency maps is described. In one or more implementations, component scores, indicative of visual characteristics established for visually-pleasing croppings, are computed for candidate image croppings using multiple different saliency maps. The visual characteristics on which a candidate image cropping is scored may be indicative of its composition quality, an extent to which it preserves content appearing in the scene, and a simplicity of its boundary. Based on the component scores, the croppings may be ranked with regard to each of the visual characteristics. The rankings may be used to cluster the candidate croppings into groups of similar croppings, such that croppings in a group are different by less than a threshold amount and croppings in different groups are different by at least the threshold amount. Based on the clustering, croppings may then be chosen, e.g., to present them to a user for selection.

IMAGE CROP SUGGESTION AND EVALUATION USING DEEP-LEARNING

Various embodiments describe using a neural network to evaluate image crops in substantially real-time. In an example, a computer system performs unsupervised training of a first neural network based on unannotated image crops, followed by a supervised training of the first neural network based on annotated image crops. Once this first neural network is trained, the computer system inputs image crops generated from images to this trained network and receives composition scores therefrom. The computer system performs supervised training of a second neural network based on the images and the composition scores. 1. A computer-implemented method associated with cropping images , the method comprising:performing, by a computer system, unsupervised training of a first neural network based on unannotated image crops, the unsupervised training comprising updating parameters of the first neural network associated with generating a composition score of an image crop within an image;performing, by the computer system upon completion of the unsupervised training, first supervised training of the first neural network based on annotated image crops, the first supervised training further updating the parameters of the first neural network;generating, by the computer system, image crops from images based on predefined crop areas;receiving, by the computer system from the first neural network, composition scores of the image crops based on inputting the image crops to the first neural network upon completion of the first supervised training;performing, by the computer system, second supervised training of a second neural network based on the images and the composition scores, the second supervised training comprising updating parameters of the second neural network associated with evaluating compositions of the predefined crop areas within the image; andproviding, by the computer system to an image application, information about a composition of the image crop within the image based ...

Guided image composition on mobile devices

Various embodiments describe facilitating real-time crops on an image. In an example, an image processing application executed on a device receives image data corresponding to a field of view of a camera of the device. The image processing application renders a major view on a display of the device in a preview mode. The major view presents a previewed image based on the image data. The image processing application receives a composition score of a cropped image from a deep-learning system. The image processing application renders a sub-view presenting the cropped image based on the composition score in a preview mode. Based on a user interaction, the image processing application renders the cropped image in the major view with the sub-view in the preview mode.

Method for Using Deep Learning for Facilitating Real-Time View Switching and Video Editing on Computing Devices

Various embodiments describe view switching of video on a computing device. In an example, a video processing application executed on the computing device receives a stream of video data. The video processing application renders a major view on a display of the computing device. The major view presents a video from the stream of video data. The video processing application inputs the stream of video data to a deep learning system and receives back information that identifies a cropped video from the video based on a composition score of the cropped video, while the video is presented in the major view. The composition score is generated by the deep learning system. The video processing application renders a sub-view on a display of the device, the sub-view presenting the cropped video. The video processing application renders the cropped video in the major view based on a user interaction with the sub-view. 1. A computer-implemented method for facilitating real-time view switching and video editing , the method comprising:receiving, by a video processing application executed on a computing device, a stream of video data;rendering, by the video processing application, a major view on a display of the computing device, the major view presenting a video from the stream of video data;inputting, by the video processing application, the stream of video data to a deep learning system;receiving, by the video processing application from the deep learning system, information that identifies a cropped video from the video based on a composition score of the cropped video, the composition score generated by the deep learning system while the video is presented in the major view, the cropped video having a same aspect ratio as the video;rendering, by the video processing application, a sub-view on the display of the computing device, the sub-view presenting the cropped video; andrendering, by video processing application, the cropped video in the major view based on a user ...

Image Zooming

Image zooming is described. In one or more implementations, zoomed croppings of an image are scored. The scores calculated for the zoomed croppings are indicative of a zoomed cropping's inclusion of content that is captured in the image. For example, the scores are indicative of a degree to which a zoomed cropping includes salient content of the image, a degree to which the salient content included in the zoomed cropping is centered in the image, and a degree to which the zoomed cropping preserves specified regions-to-keep and excludes specified regions-to-remove. Based on the scores, at least one zoomed cropping may be chosen to effectuate a zooming of the image. Accordingly, the image may be zoomed according to the zoomed cropping such that an amount the image is zoomed corresponds to a scale of the zoomed cropping. 1. A method implemented by a computing device , the method comprising:determining zoomed croppings of an image, each zoomed cropping including content captured in the image;calculating, for each zoomed cropping of the image, a plurality of scores that are indicative of inclusion of the content captured in the image, each of the zoomed croppings having a scale that corresponds to an amount the image is zoomed to result in the zoomed croppings; andchoosing one or more of the zoomed croppings having the scale based on the plurality of scores calculated for the zoomed croppings.2. A method as described in claim 1 , wherein each of the zoomed croppings has a same aspect ratio as the image.3. A method as described in claim 1 , wherein at least one of the plurality of scores is indicative of a degree to which the zoomed croppings include salient content of the image.4. A method as described in claim 1 , wherein at least one of the plurality of scores is indicative of a degree to which salient content included in the zoomed croppings is centered in the zoomed croppings.5. A method as described in claim 1 , wherein at least one of the plurality of scores is ...

Digital Image Completion Using Deep Learning

Digital image completion using deep learning is described. Initially, a digital image having at least one hole is received. This holey digital image is provided as input to an image completer formed with a framework that combines generative and discriminative neural networks based on learning architecture of the generative adversarial networks. From the holey digital image, the generative neural network generates a filled digital image having hole-filling content in place of holes. The discriminative neural networks detect whether the filled digital image and the hole-filling digital content correspond to or include computer-generated content or are photo-realistic. The generating and detecting are iteratively continued until the discriminative neural networks fail to detect computer-generated content for the filled digital image and hole-filling content or until detection surpasses a threshold difficulty. Responsive to this, the image completer outputs the filled digital image with hole-filling content in place of the holey digital image's holes. 1. In a digital medium environment to complete digital images having holes , a method implemented by a computing device , the method comprising:exposing, by the computing device, a holey digital image to an image completer having a framework that combines generative and discriminative neural networks using adversarial deep learning, the holey digital image having at least one hole comprising a group of contiguous pixels with values indicating an absence of depicted imagery; and generating proposed hole-filling digital content with the generative neural network based, in part, on the depicted imagery of the holey digital image; and', 'determining with the discriminative neural networks whether the proposed hole-filling digital content is computer-generated digital content, the proposed hole-filling digital content used as the hole-filling digital content., 'receiving, by the computing device, a filled digital image as ...

Predicting Patch Displacement Maps Using A Neural Network

Predicting patch displacement maps using a neural network is described. Initially, a digital image on which an image editing operation is to be performed is provided as input to a patch matcher having an offset prediction neural network. From this image and based on the image editing operation for which this network is trained, the offset prediction neural network generates an offset prediction formed as a displacement map, which has offset vectors that represent a displacement of pixels of the digital image to different locations for performing the image editing operation. Pixel values of the digital image are copied to the image pixels affected by the operation by: determining the vectors pixels that correspond to the image pixels affected by the image editing operation and mapping the pixel values of the image pixels represented by the determined offset vectors to the affected pixels. According to this mapping, the pixel values of the affected pixels are set, effective to perform the image editing operation. 1. In a digital medium environment to perform image editing operations involving patch matching , a method implemented by a computing device , the method comprising:receiving, by the computing device, a digital image relative to which an image editing operation is to be performed;exposing, by the computing device, the digital image to a patch matcher having an offset prediction neural network; and generating, with the offset prediction neural network, an offset prediction as a displacement map, the displacement map comprising offset vectors that represent a displacement of image pixels of the digital image for performing the image editing operation;', 'determining the offset vectors that correspond to the image pixels affected by the image editing operation;', 'mapping pixel values of the image pixels that are represented by the determined offset vectors to the affected image pixels, and', 'setting pixel values of the affected image pixels according to the ...

Real-Time Location Estimation of a Mobile Station in a Wireless Communication System

An apparatus and method is provided for real-time location estimation of a mobile station (MS) on a map of roads using non-Global Positioning Satellite location data of the MS and includes a step of determining that the MS is stationary. A next step includes assigning each raw location datum with an initial mass such that the data exhibit an attractive force therebetween. A next step includes calculating a net attractive force for all data using a distance between the data. A next step includes moving each datum a normalized step on the map in response to the net attractive force. A next step includes merging any data that are within a predefined distance on the map and adding their masses. A next step includes repeating the calculating, moving, and merging steps until there are no consecutive merges for a predetermined number of iterations. A next step includes removing any datum with a mass less than a threshold. If there is more than one merged datum left, and a total number of iterations is less than a predefined number, going to the calculating step. Otherwise, using a mean of the locations of any remaining data as a location of the mobile station. 1. A method for real-time location estimation of a mobile station in a wireless communication system on a geographic map of roads using non-Global Positioning Satellite (non-GPS) raw location data of the mobile station and depending on a mobility status of the mobile station , the method comprising the steps of: assigning each raw location datum with an initial mass such that the data exhibit an attractive force therebetween;', 'calculating a net attractive force for all data using a distance between the data;', 'moving each datum a normalized step on the map in response to the net attractive force;', 'merging any data that are within a predefined distance on the map and adding their masses;', 'repeating the calculating, moving, and merging steps until there are no consecutive merges for a predetermined number of ...

OBJECT DETECTION WITH BOOSTED EXEMPLARS

In techniques for object detection with boosted exemplars, weak classifiers of a real-adaboost technique can be learned as exemplars that are collected from example images. The exemplars are examples of an object that is detectable in image patches of an image, such as faces that are detectable in images. The weak classifiers of the real-adaboost technique can be applied to the image patches of the image, and a confidence score is determined for each of the weak classifiers as applied to an image patch of the image. The confidence score of a weak classifier is an indication of whether the object is detected in the image patch of the image based on the weak classifier. All of the confidence scores of the weak classifiers can then be summed to generate an overall object detection score that indicates whether the image patch of the image includes the object. 1. A method , comprising:applying weak classifiers of a real-adaboost technique to image patches of an image, the weak classifiers being exemplars taken from example images as examples of an object that is detectable in the image patches of the image;determining a confidence score for each of the weak classifiers as applied to an image patch of the image, the confidence score of a weak classifier being an indication of whether the object is detected in the image patch of the image based on the weak classifier; andsumming all of the confidence scores of the weak classifiers to generate an object detection score that indicates whether the image patch of the image includes the object.2. The method as recited in claim 1 , wherein:the object is representative of faces that are detectable in the image patches of the image;the exemplars are example faces taken from the example images; andthe confidence score for each of the weak classifiers is the indication of whether a face of a person is detected in the image patch of the image.3. The method as recited in claim 2 , wherein the faces are detectable in the image patches ...

Enhancement of Skin, Including Faces, in Photographs

An image processing application performs improved face exposure correction on an input image. The image processing application receives an input image having a face and ascertains a median luminance associated with a face region corresponding to the face. The image processing application determines whether the median luminance is less than a threshold luminance. If the median luminance is less than the threshold luminance, the application computes weights based on a spatial distance parameter and a similarity parameter associated with the median chrominance of the face region. The image processing application then computes a corrected luminance using the weights and applies the corrected luminance to the input image. The image processing application can also perform improved face color correction by utilizing stylization-induced shifts in skin tone color to control how aggressively stylization is applied to an image. 1. In a digital medium environment including an image processing application that performs face exposure correction on an input image , an improved face exposure correction method implemented by the image processing application , the method comprising:receiving an input image including a depiction of a face;ascertaining a median luminance associated with a face region corresponding to the depiction of the face;determining whether the median luminance is less than a threshold luminance;responsive to the median luminance being less than the threshold luminance, computing weights based on a spatial distance parameter and a similarity parameter associated with a median chrominance of the face region;computing a corrected luminance using the weights; andapplying the corrected luminance to the input image;wherein the method is performed by a computing device executing the image processing application.2. A method as described in claim 1 , wherein the spatial distance parameter corresponds to how close a particular pixel in the input image is to a center of the ...

DEEP SALIENT CONTENT NEURAL NETWORKS FOR EFFICIENT DIGITAL OBJECT SEGMENTATION

Systems, methods, and non-transitory computer-readable media are disclosed for segmenting objects in digital visual media utilizing one or more salient content neural networks. In particular, in one or more embodiments, the disclosed systems and methods train one or more salient content neural networks to efficiently identify foreground pixels in digital visual media. Moreover, in one or more embodiments, the disclosed systems and methods provide a trained salient content neural network to a mobile device, allowing the mobile device to directly select salient objects in digital visual media utilizing a trained neural network. Furthermore, in one or more embodiments, the disclosed systems and methods train and provide multiple salient content neural networks, such that mobile devices can identify objects in real-time digital visual media feeds (utilizing a first salient content neural network) and identify objects in static digital images (utilizing a second salient content neural network). 1. A non-transitory computer readable medium storing instructions thereon that , when executed by at least one processor , cause a computer system to:receive, by a mobile device, a digital image portraying one or more salient objects;access a salient content neural network on the mobile device, wherein the salient content neural network is trained by utilizing the salient content neural network to predict foreground pixels of a training digital image and comparing ground truth foreground pixels of the training digital image with the predicted foreground pixels of the training digital image;identify, by the mobile device, the one or more salient objects portrayed in the digital image by applying the salient content neural network to the digital image; andgenerate a modified digital image based on the identified one or more salient objects portrayed in the digital image.2. The non-transitory computer readable medium of claim 1 , wherein the digital image is part of a real-time ...

Neural Network Patch Aggregation and Statistics

Neural network patch aggregation and statistical techniques are described. In one or more implementations, patches are generated from an image, e.g., randomly, and used to train a neural network. An aggregation of outputs of patches processed by the neural network may be used to label an image using an image descriptor, such as to label aesthetics of the image, classify the image, and so on. In another example, the patches may be used by the neural network to calculate statistics describing the patches, such as to describe statistics such as minimum, maximum, median, and average of activations of image characteristics of the individual patches. These statistics may also be used to support a variety of functionality, such as to label the image as described above. 1. A method comprising:generating a plurality of patches from an image by one or more computing devices;calculating activations of a plurality of image characteristics for each of the plurality of patches using a neural network;sorting the calculated activations according to respective said image characteristics by the one or more computing devices;aggregating the sorted activations for each of the plurality of image characteristics by the one or more computing devices; andlabeling the image using an image descriptor based on the aggregated and sorted activations by the one or more computing devices, the image descriptor indicative of a confidence that the image has an image attribute involving the plurality of image characteristics.2. A method as described in claim 1 , wherein the generating is performed such that the plurality of patches are selected randomly from the image in an order-less manner.3. A method as described in claim 1 , wherein the plurality of patches are taken from the image without down-sampling the image.4. A method as described in claim 1 , wherein the aggregating includes ranking the calculated activations claim 1 , one to another claim 1 , for the image characteristic.5. A method as ...

Object Segmentation, Including Sky Segmentation

A digital medium environment includes an image processing application that performs object segmentation on an input image. An improved object segmentation method implemented by the image processing application comprises receiving an input image that includes an object region to be segmented by a segmentation process, processing the input image to provide a first segmentation that defines the object region, and processing the first segmentation to provide a second segmentation that provides pixel-wise label assignments for the object region. In some implementations, the image processing application performs improved sky segmentation on an input image containing a depiction of a sky. 1. In a digital medium environment including an image processing application that performs object segmentation on an input image , an improved object segmentation method implemented by the image processing application , the method comprising:receiving an input image that includes an object region to be segmented; parsing the input image to provide a probability mask which classifies individual pixels in the input image;', 'determining, from a database, multiple images which have layouts at least similar to a layout of the input image, wherein the multiple images include respective masks;', 'processing the respective masks to provide a weighted average mask; and', 'combining the probability mask and the weighted average mask to provide the first segmentation., 'processing the input image to provide a first segmentation that defines the object region, said processing comprising2. A method as described in claim 1 , wherein the object region to be segmented comprises a depiction of a sky.3. A method as described in claim 1 , wherein parsing the input image to provide the probability mask comprises parsing the input image using a Conditional Random Field to provide the probability mask.4. A method as described in claim 1 , wherein determining the multiple images from the database comprises ...

ROBUST TRACKING OF OBJECTS IN VIDEOS

The present disclosure is directed toward systems and methods for tracking objects in videos. For example, one or more embodiments described herein utilize various tracking methods in combination with an image search index made up of still video frames indexed from a video. One or more embodiments described herein utilize a backward and forward tracking method that is anchored by one or more key frames in order to accurately track an object through the frames of a video, even when the video is long and may include challenging conditions. 1. In a digital environment for tracking objects in videos , a method of identifying objects in videos comprising:receiving a video;extracting a plurality of video frames from the video;generating an image search index from the plurality of video frames;receiving an indication of a query object within one or more key frames of the plurality of video frames; anda step for identifying the query object in video frames of the plurality of video frames based on the one or more key frames.2. The method as recited in claim 1 , further comprising:redacting the query object from the video frames in which the query object is identified; andgenerating a redacted video by merging the video frames with a remainder of the plurality of video frames.3. The method as recited in claim 1 , further comprising a step for generating auxiliary key frames claim 1 , and wherein the one or more key frames comprises the key frame and a plurality of auxiliary key frames.4. The method as recited in claim 3 , wherein the step for identifying the query object in video frames of the plurality of video frames based on the one or more key frames comprises:weighting a plurality of similarity scores for candidate query objects in a candidate video frame using a time decay function; andselecting the candidate query object for the candidate video frame that has the maximum weighted similarity score.5. The method as recited in claim 1 , wherein the step for identifying ...

Image Color and Tone Style Transfer

In embodiments of image color and tone style transfer, a computing device implements an image style transfer algorithm to generate a modified image from an input image based on a color style and a tone style of a style image. A user can select the input image that includes color features, as well as select the style image that includes an example of the color style and the tone style to transfer to the input image. A chrominance transfer function can then be applied to transfer the color style to the input image, utilizing a covariance of an input image color of the input image to control modification of the input image color. A luminance transfer function can also be applied to transfer the tone style to the input image, utilizing a tone mapping curve based on a non-linear optimization to estimate luminance parameters of the tone mapping curve. 1. A method to transfer a color and tone style of a style image to an input image , the method comprising:receiving a selection of the input image that includes color features;receiving a selection of the style image that includes an example of the color and tone style to transfer to the input image;generating a modified image from the input image based on the color and tone style of the style image, the generating comprising:applying a chrominance transfer function to transfer the color style of the style image to the input image, the chrominance transfer function implementing a covariance of an input image color of the input image to control modification of the input image color; andapplying a luminance transfer function to transfer the tone style of the style image to the input image, the luminance transfer function implementing a tone mapping curve based on a non-linear optimization to estimate luminance parameters of the tone mapping curve.2. The method as recited in claim 1 , wherein said generating the modified image based on the color and tone style of the style image without creating visual artifacts of the color ...

RECOGNIZING UNKNOWN PERSON INSTANCES IN AN IMAGE GALLERY

Methods and systems for recognizing people in images with increased accuracy are disclosed. In particular, the methods and systems divide images into a plurality of clusters based on common characteristics of the images. The methods and systems also determine an image cluster to which an image with an unknown person instance most corresponds. One or more embodiments determine a probability that the unknown person instance is each known person instance in the image cluster using a trained cluster classifier of the image cluster. Optionally, the methods and systems determine context weights for each combination of an unknown person instance and each known person instance using a conditional random field algorithm based on a plurality of context cues associated with the unknown person instance and the known person instances. The methods and systems calculate a contextual probability based on the cluster-based probabilities and context weights to identify the unknown person instance. 1. A method of identifying an unknown person instance in an image using cluster based-person recognition comprising:dividing, by at least one processor, images of an image gallery into a plurality of image clusters, each image cluster comprising a plurality of images from the image gallery that share one or more common characteristics;training, by the at least one processor, a cluster classifier for each image cluster of the plurality of image clusters based on a plurality of known person instances;determining, by the at least one processor, an image cluster to which the image most corresponds based on one or more characteristics of the image and common characteristics of the plurality of image clusters; anddetermining, by the at least one processor, a probability that the unknown person instance is each known person instance in the image cluster using the cluster classifier of the image cluster.2. The method as recited in claim 1 , wherein determining the image cluster to which the image ...

Convolutional Neural Network Using a Binarized Convolution Layer

A convolutional neural network is trained to analyze input data in various different manners. The convolutional neural network includes multiple layers, one of which is a convolution layer that performs a convolution, for each of one or more filters in the convolution layer, of the filter over the input data. The convolution includes generation of an inner product based on the filter and the input data. Both the filter of the convolution layer and the input data are binarized, allowing the inner product to be computed using particular operations that are typically faster than multiplication of floating point values. The possible results for the convolution layer can optionally be pre-computed and stored in a look-up table. Thus, during operation of the convolutional neural network, rather than performing the convolution on the input data, the pre-computed result can be obtained from the look-up table

OBJECT DETECTION USING CASCADED CONVOLUTIONAL NEURAL NETWORKS

Different candidate windows in an image are identified, such as by sliding a rectangular or other geometric shape of different sizes over an image to identify portions of the image (groups of pixels in the image). The candidate windows are analyzed by a set of convolutional neural networks, which are cascaded so that the input of one convolutional neural network layer is based on the input of another convolutional neural network layer. Each convolutional neural network layer drops or rejects one or more candidate windows that the convolutional neural network layer determines does not include an object (e.g., a face). The candidate windows that are identified as including an object (e.g., a face) are analyzed by another one of the convolutional neural network layers. The candidate windows identified by the last of the convolutional neural network layers are the indications of the objects (e.g., faces) in the image. 1. A method comprising:identifying multiple candidate windows in an image, each candidate window including a group of pixels of the image, the multiple candidate windows including overlapping candidate windows;identifying one or more of the multiple candidate windows that include an object, the identifying including analyzing the multiple candidate windows using cascaded convolutional neural networks, the cascaded convolutional neural networks including multiple cascade layers, each cascade layer comprising a convolutional neural network, the multiple cascade layers including a first cascade layer that analyzes the identified multiple candidate windows, a second cascade layer that analyzes ones of the multiple candidate windows identified by the first cascade layer as including an object, and a third cascade layer that analyzes ones of the multiple candidate windows identified by the second cascade layer as including an object; andoutputting, as an indication of one or more objects in the image, an indication of one or more of the multiple candidate ...

NEURAL NETWORK BASED FACE DETECTION AND LANDMARK LOCALIZATION

Approaches are described for determining facial landmarks in images. An input image is provided to at least one trained neural network that determines a face region (e.g., bounding box of a face) of the input image and initial facial landmark locations corresponding to the face region. The initial facial landmark locations are provided to a 3D face mapper that maps the initial facial landmark locations to a 3D face model. A set of facial landmark locations are determined from the 3D face model. The set of facial landmark locations are provided to a landmark location adjuster that adjusts positions of the set of facial landmark locations based on the input image. The input image is presented on a user device using the adjusted set of facial landmark locations. 1. A computer-performed method for determining facial landmarks in images , comprising:generating adjustments to a face region of an input image using a trained joint calibration and alignment neural network;identifying initial facial landmark locations corresponding to the adjustments using the trained joint calibration and alignment neural network;generating refined facial landmark locations from the initial facial landmark locations; andcausing presentation of the input image on a user device using the refined facial landmark locations.2. The method of claim 1 , wherein the adjustments to the face region and the initial facial landmark locations are generated from a common fully-connected layer of the trained joint calibration and alignment neural network.3. The method of claim 1 , wherein the landmark location refiner comprises a landmark location adjuster that adjusts positions of a set of facial landmark locations corresponding to the initial facial landmark locations.4. The method of claim 1 , wherein the generating refined facial landmark locations from the initial facial landmark locations comprises:mapping the initial facial landmark locations to a 3D face model;determining a set of facial landmark ...

Object Detection In Images

In implementations of object detection in images, object detectors are trained using heterogeneous training datasets. A first training dataset is used to train an image tagging network to determine an attention map of an input image for a target concept. A second training dataset is used to train a conditional detection network that accepts as conditional inputs the attention map and a word embedding of the target concept. Despite the conditional detection network being trained with a training dataset having a small number of seen classes (e.g., classes in a training dataset), it generalizes to novel, unseen classes by concept conditioning, since the target concept propagates through the conditional detection network via the conditional inputs, thus influencing classification and region proposal. Hence, classes of objects that can be detected are expanded, without the need to scale training databases to include additional classes. 1. In a digital medium environment to detect objects in images , an object detection method implemented by a computing device , the object detection method comprising:obtaining, by the computing device, an input image and a word-based concept;generating, by an image tagging network of the computing device, an attention map based on the input image and the word-based concept, the attention map including pixels indicating presence values for the word-based concept within the input image;generating, by the computing device, a word embedding based on the word-based concept, the word embedding describing relationships between the word-based concept and different word-based concepts;providing, by the computing device, the word embedding and the attention map to respective layers of a conditional detection network; anddetecting, by the conditional detection network of the computing device, at least one region of the input image based on the word embedding and the attention map, the at least one region including a respective object corresponding ...

OBJECT DETECTION VIA VALIDATION WITH VISUAL SEARCH

One exemplary embodiment involves receiving, at a computing device comprising a processor, a test image having a candidate object and a set of object images detected to depict a similar object as the test image. The embodiment involves localizing the object depicted in each one of the object images based on the candidate object in the test image to determine a location of the object in each respective object image and then generating a validation score for the candidate object in the test image based at least in part on the determined location of the object in the respective object image and known location of the object in the same respective object image. The embodiment also involves computing a final detection score for the candidate object based on the validation score that indicates a confidence level that the object in the test image is located as indicated by the candidate object. 1. A method comprising:identifying a candidate region of a test image as potentially depicting an instance of an object;identifying a first region in an object image that is similar to the candidate region of the test image, the object image being different from the test image;performing a comparison of the first region with a second region of the object image, the second region known to actually depict the instance of the object; anddetermining a validation score for the candidate region based on the comparison,wherein identifying the candidate region, identifying the first region, performing the comparison, and determining the validation score are performed by a processor of a computer device executing instructions.2. The method of claim 21 , wherein the validation score is determined based on multiple comparisons involving multiple object images claim 21 , each comparison comprising a respective localized region based on the candidate region and a respective known region at which a respective instance of the object is known.3. The method of claim 2 , wherein the multiple object ...

Attribute recognition via visual search

One exemplary embodiment involves identifying feature matches between each of a plurality of object images and a test image, each feature matches between a feature of a respective object image and a matching feature of the test image, wherein there is a spatial relationship between each respective object image feature and a test image feature, and wherein the object depicted in the test image comprises a plurality of attributes. Additionally, the embodiment involves estimating, for each attribute in the test image, an attribute value based at least in part on information stored in a metadata associated with each of the object images.

LANDMARK LOCALIZATION VIA VISUAL SEARCH

One exemplary embodiment involves identifying feature matches between each of a plurality of object images and a test image, each of the feature matches between a feature of a respective object image and a matching feature of the test image, wherein there is a spatial relationship between each respective object image feature and a first landmark of the object image, the first landmark at a known location in the object image. The embodiment additionally involves estimating a plurality of locations for a second landmark for the test image, the estimated locations based at least in part on the feature matches and the spatial relationships, and estimating a final location for the second landmark from the plurality of locations for the second landmark for the test image. 1. A computer-implemented method comprising:receiving, at a computing device comprising a processor, a test image and a plurality of object images detected to depict an object similar to the test image;receiving a plurality of landmark indicators, each landmark indicator indicating a location of a landmark in the object depicted in a respective one of the object image;for each object image, determining a location of a test image landmark that corresponds with the respective landmark indicator in the object image; andcomputing a final test image landmark location based on the determined locations of the test image landmarks.2. The computer-implemented method of claim 1 , wherein the landmark indicators of the object images are manually identified to indicate a location of the landmark.3. The computer-implemented method of claim 1 , wherein generating the similarity voting map comprises identifying matching features between the test image and one of the landmarks of the object image indicated by the landmark indicator.4. The computer-implemented method of claim 3 , wherein the similarity voting map is generated by applying a weight to a plurality of matching features that are within a predetermined ...

OBJECT DETECTION VIA VISUAL SEARCH

One exemplary embodiment involves receiving a test image generating, by a plurality of maps for the test image based on a plurality of object images. Each of the object images comprises an object of a same object type, e.g., each comprising a different face. Each of the plurality of maps is generated to provide information about the similarity of at least a portion of a respective object image to each of a plurality of portions of the test image. The exemplary embodiment further comprises detecting a test image object within the test image based at least in part on the plurality of maps. 1. A computer-implemented method comprising:receiving, at a computing device comprising a processor, a test image;generating, by the processor, a plurality of maps for the test image based on a plurality of object images, each of the plurality of maps generated to provide information about a similarity of an object portion of a respective object image to each of a plurality of portions of the test image; anddetecting a test image object within the test image based at least in part on the plurality of maps.2. The computer-implemented method of claim 1 , wherein detecting the test image object comprises determining a location of the test image object within the test image based at least in part on the plurality of maps.3. The computer-implemented method of claim 1 , wherein detecting the test image object comprises:gating each of the plurality of maps to generate a plurality of gated maps, wherein gating each map comprises subtracting an object-image-specific similarity threshold from each similarity score represented in the map and removing negative results to produce a gated map;aggregating the plurality of gated maps to generate an aggregate map; andusing the aggregate map to detect the test image object within the test image.4. The computer-implemented method of wherein detecting the test image object comprises:aggregating the maps of the plurality of maps to generate an aggregate ...

High dynamic range illumination estimation

Systems and techniques for estimating illumination from a single image are provided. An example system may include a neural network. The neural network may include an encoder that is configured to encode an input image into an intermediate representation. The neural network may also include an intensity decoder that is configured to decode the intermediate representation into an output light intensity map. An example intensity decoder is generated by a multi-phase training process that includes a first phase to train a light mask decoder using a set of low dynamic range images and a second phase to adjust parameters of the light mask decoder using a set of high dynamic range image to generate the intensity decoder.

FRAME SELECTION BASED ON A TRAINED NEURAL NETWORK

Various embodiments describe frame selection based on training and using a neural network. In an example, the neural network is a convolutional neural network trained with training pairs. Each training pair includes two training frames from a frame collection. The loss function relies on the estimated quality difference between the two training frames. Further, the definition of the loss function varies based on the actual quality difference between these two frames. In a further example, the neural network is trained by incorporating facial heatmaps generated from the training frames and facial quality scores of faces detected in the training frames. In addition, the training involves using a feature mean that represents an average of the features of the training frames belonging to the same frame collection. Once the neural network is trained, a frame collection is input thereto and a frame is selected based on generated quality scores. 1. A computer-implemented method , comprising:providing a collection of images to a neural network, the neural network trained using training data comprising training images and associated training labels, wherein a training label associated with a training image indicates a quality measure for the training image, the neural network having associated parameters resulting from minimization of a loss function based on a first quality difference and a second quality difference, wherein the first quality difference is based on a training label associated with a first training image in the training images and a training label associated with a second training image in the training images, and wherein the second quality difference is based on an estimation of quality of the first training image and an estimation of quality of the second training image generated by the neural network;generating, using the neural network, an estimation of quality for each image in the collection of images; andbased upon the estimations of quality generated ...

Neural Network Image Curation Control

Neural network image curation techniques are described. In one or more implementations, curation is controlled of images that represent a repository of images. A plurality of images of the repository are curated by one or more computing devices to select representative images of the repository. The curation includes calculating a score based on image and face aesthetics, jointly, for each of the plurality of images through processing by a neural network, ranking the plurality of images based on respective said scores, and selecting one or more of the plurality of images as one of the representative images of the repository based on the ranking and a determination that the one or more said images are not visually similar to images that have already been selected as one of the representative images of the repository. 1. A method to control curation of images that represent a repository of images , the method comprising: calculating a score based on image and face aesthetics, jointly, for each of the plurality of images through processing by a neural network;', 'ranking the plurality of images based on respective said scores; and', 'selecting one or more of the plurality of images as one of the representative images of the repository based on the ranking and a determination that the one or more said images are not visually similar to images that have already been selected as one of the representative images of the repository., 'curating a plurality of images of the repository by one or more computing devices to select representative images of the repository, the curating including2. A method as described in claim 1 , wherein the calculating of the score based on image and face aesthetics claim 1 , jointly claim 1 , for each of the plurality of images includes:generating a plurality of patches from the each said image;detecting activations of a plurality of image characteristics for each of the plurality of patches using the neural network, the plurality of image ...

Image Cropping Suggestion Using Multiple Saliency Maps

Image cropping suggestion using multiple saliency maps is described. In one or more implementations, component scores, indicative of visual characteristics established for visually-pleasing croppings, are computed for candidate image croppings using multiple different saliency maps. The visual characteristics on which a candidate image cropping is scored may be indicative of its composition quality, an extent to which it preserves content appearing in the scene, and a simplicity of its boundary. Based on the component scores, the croppings may be ranked with regard to each of the visual characteristics. The rankings may be used to cluster the candidate croppings into groups of similar croppings, such that croppings in a group are different by less than a threshold amount and croppings in different groups are different by at least the threshold amount. Based on the clustering, croppings may then be chosen, e.g., to present them to a user for selection. 1. A method implemented by a computing device , the method comprising:obtaining input to initiate cropping of a scene;computing multiple different saliency maps from an image of the scene; and composition quality of the multiple candidate croppings, the assessment of the composition quality performed using a first combination of the multiple different saliency maps;', 'degrees of preservation of content appearing in the scene for the multiple candidate croppings, the assessment of the degrees of preservation of content performed using a second combination of the multiple different saliency maps; and', 'boundary simplicity of the multiple candidate croppings, the assessment of the boundary simplicity performed using a third combination of the multiple different saliency maps., 'generating one or more suggested image croppings of the scene based on rankings assigned to multiple candidate croppings that reflect an assessment of2. A method as described in claim 1 , wherein the first combination of saliency maps includes:a ...

Depth-of-Field Blur Effects Generating Techniques

Techniques of generating depth-of-field blur effects on digital images by digital effect generation system of a computing device are described. The digital effect generation system is configured to generate depth-of-field blur effects on objects based on focal depth value that defines a depth plane in the digital image and a aperture value that defines an intensity of blur effect applied to the digital image. The digital effect generation system is also configured to improve the accuracy with which depth-of-field blur effects are generated by performing up-sampling operations and implementing a unique focal loss algorithm that minimizes the focal loss within digital images effectively. 1. In a digital image generating environment , a method implemented by a computing device to generate a digital image with a depth-of-field blur effect , the method comprising:receiving, by the computing device, user inputs specifying a focal depth value that defines a depth plane within the digital image and an aperture value describing an intensity of the depth-of-field blur effect;generating, by the computing device, a down-sampled digital image from the digital image;generating, by the computing device, a depth map from the down-sampled digital image; andgenerating, by the computing device, a down-sampled digital image with the depth-of-field blur effect based on the focal depth value, the aperture value, and the depth map.2. The method as described in claim 1 , wherein the generating of the down-sampled digital image with the depth-of-field blur effect comprises:generating, by the computing device, a feature map from the depth map of the down-sampled digital image using a convolutional neural network;generating, by the computing device, a kernel tensor using the focal depth value, the aperture value, and the depth map; andapplying, by the computing device, the kernel tensor to the feature map of the down-sampled digital image.3. The method as described in claim 1 , wherein the ...

Joint Training Technique for Depth Map Generation

Joint training technique for depth map generation implemented by depth prediction system as part of a computing device is described. The depth prediction system is configured to generate a candidate feature map from features extracted from training digital images, generate a candidate segmentation map and a candidate depth map from the generated candidate feature map, and jointly train portions of the depth prediction system using a loss function. Consequently, depth prediction system is able to generate a depth map that identifies depths of objects using ordinal depth information and accurately delineates object boundaries within a single digital image. 1. In a digital medium training environment to train a neural network to generate a depth map , a method implemented by at least one computing device , the method comprising:generating, by the at least one computing device, at least one candidate feature map by extracting features from training digital images using a feature extraction module of a neural network;generating, by the at least one computing device, a candidate segmentation map from the at least one candidate feature map using a segmentation module of the neural network;generating, by the at least one computing device, a candidate depth map from the at least one candidate feature map using a depth module of the neural network; andjointly training, by the at least one computing device, the segmentation module and the depth module of the neural network using a loss function based on the candidate segmentation map, the candidate depth map, a respective ground truth segmentation map, and a respective ground truth depth map.2. The method as described in claim 1 , wherein the generating of the at least one candidate feature map includes:generating at least one initial candidate feature map by extracting the features from the training digital images using an encoder of the feature extraction module of the neural network; andUp-sampling the at least one initial ...

Digital Image Completion Using Deep Learning

Digital image completion using deep learning is described. Initially, a digital image having at least one hole is received. This holey digital image is provided as input to an image completer formed with a framework that combines generative and discriminative neural networks based on learning architecture of the generative adversarial networks. From the holey digital image, the generative neural network generates a filled digital image having hole-filling content in place of holes. The discriminative neural networks detect whether the filled digital image and the hole-filling digital content correspond to or include computer-generated content or are photo-realistic. The generating and detecting are iteratively continued until the discriminative neural networks fail to detect computer-generated content for the filled digital image and hole-filling content or until detection surpasses a threshold difficulty. Responsive to this, the image completer outputs the filled digital image with hole-filling content in place of the holey digital image's holes. 1. In a digital medium environment to train an image completion framework to complete images having holes , a method implemented by a computing device , the method comprising:receiving filled images from the image completion framework, the image completion framework formed by combining a generative neural network with global and local discriminative neural networks, the filled images having hole filling generated by the generative neural network to fill holes, and the filled images and the hole filling detected to be photo-realistic by the global and local discriminative neural networks, respectively;comparing the filled images to respective training images prior to introduction of the holes into the training images and based on at least one loss function; andadjusting parameters of the generative neural network or the global and local discriminative neural networks based on the comparing.2. A method as described in claim ...

Neural Network Image Curation Control

Neural network image curation techniques are described. In one or more implementations, curation is controlled of images that represent a repository of images. A plurality of images of the repository are curated by one or more computing devices to select representative images of the repository. The curation includes calculating a score based on image and face aesthetics, jointly, for each of the plurality of images through processing by a neural network, ranking the plurality of images based on respective said scores, and selecting one or more of the plurality of images as one of the representative images of the repository based on the ranking and a determination that the one or more said images are not visually similar to images that have already been selected as one of the representative images of the repository. 1. A method to control curation of images by a computing device that represent a repository of images , the method comprising: calculating a score based on at least two of image aesthetic, face aesthetic, visual semantic, or image quality, jointly, for each of the plurality of images through processing by a neural network;', 'ranking the plurality of images based on respective said scores; and', 'selecting one or more of the plurality of images as one of the representative images of the repository based on the ranking and a determination that the one or more said images are not visually similar to images that have already been selected as one of the representative images of the repository., 'curating, by the computing device, a plurality of images of the to select representative images of the repository, the curating including2. A method as described in claim 1 , wherein the calculating of the score for each of the plurality of images includes:generating a plurality of patches from the each said image;detecting activations of a plurality of image characteristics for each of the plurality of patches using the neural network; andthe calculating of the score ...

Image Cropping Suggestion

Image cropping suggestion is described. In one or more implementations, multiple croppings of a scene are scored based on parameters that indicate visual characteristics established for visually pleasing croppings. The parameters may include a parameter that indicates composition quality of a candidate cropping, for example. The parameters may also include a parameter that indicates whether content appearing in the scene is preserved and a parameter that indicates simplicity of a boundary of a candidate cropping. Based on the scores, image croppings may be chosen, e.g., to present the chosen image croppings to a user for selection. To choose the croppings, they may be ranked according to the score and chosen such that consecutively ranked croppings are not chosen. Alternately or in addition, image croppings may be chosen that are visually different according to scores which indicate those croppings have different visual characteristics. 1. A method implemented by a computing device , the method comprising:scoring candidate image croppings of a scene according to a plurality of image cropping parameters indicative of visual characteristics established for visually pleasing croppings, at least some of the visual characteristics established by analyzing a collection of images that are predefined as being visually pleasing; andchoosing one or more image croppings from the candidate image croppings based on the scoring of the candidate image croppings according to the plurality of image cropping parameters.2. A method as described in claim 1 , wherein at least one of the image cropping parameters indicates a composition quality of the candidate image croppings claim 1 , the composition quality determined based on a comparison of the candidate image croppings to composition properties derived from well-composed images.3. A method as described in claim 1 , wherein at least one of the image cropping parameters is indicative of whether at least a portion of content appearing ...

Cropping Boundary Simplicity

Cropping boundary simplicity techniques are described. In one or more implementations, multiple candidate croppings of a scene are generated. For each of the candidate croppings, a score is calculated that is indicative of a boundary simplicity for the candidate cropping. To calculate the boundary simplicity, complexity of the scene along a boundary of a respective candidate cropping is measured. The complexity is measured, for instance, using an average gradient, an image edge map, or entropy along the boundary. Values indicative of the complexity may be derived from the measuring. The candidate croppings may then be ranked according to those values. Based on the scores calculated to indicate the boundary simplicity, one or more of the candidate croppings may be chosen e.g., to present the chosen croppings to a user for selection. 1. A method implemented by a computing device , the method comprising:calculating, for candidate croppings of a scene, scores indicative of a boundary simplicity for each of the candidate croppings; andchoosing one or more croppings from the candidate croppings based, at least in part, on the scores calculated for the candidate croppings that are indicative of the boundary simplicity.2. A method as described in claim 1 , wherein calculating the scores indicative of the boundary simplicity includes:measuring complexity of the scene along a boundary of each respective cropping of the candidate croppings to derive a value indicative of the complexity measured for the boundary; andranking the candidate croppings one to another according to values derived for each respective cropping.3. A method as described in claim 2 , wherein the complexity of the scene along the boundary of each respective cropping is measured using at least one of an average gradient of the boundary claim 2 , an image edge map claim 2 , or entropy along the boundary.4. A method as described in claim 2 , wherein calculating the scores indicative of the boundary simplicity ...

Event Image Curation

In embodiments of event image curation, a computing device includes memory that stores a collection of digital images associated with a type of event, such as a digital photo album of digital photos associated with the event, or a video of image frames and the video is associated with the event. A curation application implements a convolutional neural network, which receives the digital images and a designation of the type of event. The convolutional neural network can then determine an importance rating of each digital image within the collection of the digital images based on the type of the event. The importance rating of a digital image is representative of an importance of the digital image to a person in context of the type of the event. The convolutional neural network generates an output of representative digital images from the collection based on the importance rating of each digital image. 1. A method for event image curation , the method comprising:receiving a collection of digital images associated with more than one type of event;determining by unsupervised feature learning from the collection of digital images, the types of events and an importance rating of each digital image within each respective type of event, the importance rating of a digital image representative of an importance of the digital image in context of coverage and diversity representing the type of the event; andgenerating an output of representative digital images from the collection based on the importance rating of each digital image in the context of the respective type of event associated with the digital image.2. The method as recited in claim 1 , wherein the importance rating of a digital image is representative of the coverage and diversity representing the type of the event without consideration of image quality of the digital image.3. The method as recited in claim 1 , wherein:the collection of the digital images is a digital photo album of digital photos that are ...

Predicting Patch Displacement Maps Using A Neural Network

Predicting patch displacement maps using a neural network is described. Initially, a digital image on which an image editing operation is to be performed is provided as input to a patch matcher having an offset prediction neural network. From this image and based on the image editing operation for which this network is trained, the offset prediction neural network generates an offset prediction formed as a displacement map, which has offset vectors that represent a displacement of pixels of the digital image to different locations for performing the image editing operation. Pixel values of the digital image are copied to the image pixels affected by the operation. 1. In a digital medium environment to train a patch-matching framework having an offset prediction neural network to perform image editing operations involving patch matching on digital images , a method implemented by a computing device , the method comprising:modifying, by the computing device, regions of training images based on an image editing operation the offset prediction neural network is being trained to support;exposing, by the computing device, the training images to the patch-matching framework;receiving, by the computing device, edited digital images from the patch-matching framework, the edited digital images generated by setting pixel values for image pixels of the training images according to offset predictions formed as displacement maps generated by the offset prediction neural network;comparing, by the computing device, the edited digital images to modified training images based on at least one loss function and differentiable sampling of the generated offset predictions; andadjusting, by the computing device and based on the comparing, parameters of the offset prediction neural network used in operation to generate the offset predictions.2. A method as described in claim 1 , wherein the comparing includes extracting patches from the edited digital images and the modified training ...

Category Histogram Image Representation

In techniques for category histogram image representation, image segments of an input image are generated and bounding boxes are selected that each represent a region of the input image, where each of the bounding boxes include image segments of the input image. A saliency map of the input image can also be generated. A bounding box is applied as a query on an images database to determine database image regions that match the region of the input image represented by the bounding box. The query can be augmented based on saliency detection of the input image region that is represented by the bounding box, and a query result is a ranked list of the database image regions. A category histogram for the region of the input image is then generated based on category labels of each of the database image regions that match the input image region. 1. A method , comprising:selecting bounding boxes that each represent a region of an input image, each of the bounding boxes comprising one or more image segments of the input image;applying a bounding box as a query on an images database to determine database image regions that match the region of the input image represented by the bounding box;receiving a query result of the database image regions that match the region of the input image; andgenerating a category histogram for the region of the input image based on category labels of each of the database image regions that match the region of the input image.2. The method as recited in claim 1 , wherein the category histogram for the region of the input image represents a probability distribution that the region is correctly labeled in the input image.3. The method as recited in claim 1 , further comprising:generating a saliency map of the input image; andaugmenting the query on the images database based on saliency detection of the region of the input image that is represented by the bounding box.4. The method as recited in claim 1 , further comprising:receiving the query result ...

IDENTIFYING VISUALLY SIMILAR DIGITAL IMAGES UTILIZING DEEP LEARNING

The present disclosure relates to systems, methods, and non-transitory computer readable media for utilizing a deep neural network-based model to identify similar digital images for query digital images. For example, the disclosed systems utilize a deep neural network-based model to analyze query digital images to generate deep neural network-based representations of the query digital images. In addition, the disclosed systems can generate results of visually-similar digital images for the query digital images based on comparing the deep neural network-based representations with representations of candidate digital images. Furthermore, the disclosed systems can identify visually similar digital images based on user-defined attributes and image masks to emphasize specific attributes or portions of query digital images. 1. A computer-implemented method for identifying digital images based on visual similarity comprising:receiving a search request comprising a query digital image and an indication of an area of the query digital image to emphasize;generating a deep neural network-based representation of the query digital image that weights deep features corresponding to the area of the query digital image to emphasize;identifying, from a digital image database, one or more digital images similar to the query digital image that emphasize the area of the query digital image based on the deep neural network-based representation of the query digital image; andproviding, in response to the search request, the one or more digital images similar to the query digital image that emphasize the area of the query digital image.2. The computer-implemented method of claim 1 , wherein receiving the indication of the area of the query digital image to emphasize comprises receiving an image mask that defines the area of the query digital image to emphasize.3. The computer-implemented method of claim 1 , wherein generating the deep neural network-based representation comprises: ...

UTILIZING DEEP LEARNING FOR BOUNDARY-AWARE IMAGE SEGMENTATION

Systems and methods are disclosed for segmenting a digital image to identify an object portrayed in the digital image from background pixels in the digital image. In particular, in one or more embodiments, the disclosed systems and methods use a first neural network and a second neural network to generate image information used to generate a segmentation mask that corresponds to the object portrayed in the digital image. Specifically, in one or more embodiments, the disclosed systems and methods optimize a fit between a mask boundary of the segmentation mask to edges of the object portrayed in the digital image to accurately segment the object within the digital image. 1. In a digital medium environment for editing digital visual media , a computer-implemented method of using deep learning to segment objects from the digital visual media , the computer-implemented method comprising:generating a probability map for the input image, wherein the probability map indicates object pixels predicted to correspond to the object;generating a boundary map for the input image, wherein the boundary map indicates edge pixels predicted to correspond to edges of the object;combining the probability map and the boundary map; andgenerating a segmentation mask that indicates a boundary of the object by iteratively adjusting a mask boundary of the object based on the combination of the boundary map and the probability map.2. The method of claim 1 , further comprising identifying a cropped portion of the input image comprising the object utilizing an object detection algorithm.3. The method of claim 2 , wherein:generating the probability map comprises utilizing a first de-convolutional neural network to process the cropped portion of the input image to identify the edge pixels within the cropped portion of the input image; andgenerating the boundary map comprises utilizing a second de-convolutional neural network to process the cropped portion of the input image to identify the boundary ...

ACCURATE TAG RELEVANCE PREDICTION FOR IMAGE SEARCH

Embodiments of the present invention provide an automated image tagging system that can predict a set of tags, along with relevance scores, that can be used for keyword-based image retrieval, image tag proposal, and image tag auto-completion based on user input. Initially, during training, a clustering technique is utilized to reduce cluster imbalance in the data that is input into a convolutional neural network (CNN) for training feature data. In embodiments, the clustering technique can also be utilized to compute data point similarity that can be utilized for tag propagation (to tag untagged images). During testing, a diversity based voting framework is utilized to overcome user tagging biases. In some embodiments, bigram re-weighting can down-weight a keyword that is likely to be part of a bigram based on a predicted tag set. 1. A computer-implemented method for training classifiers to tag images , the method comprising:receiving a set of input data including images and corresponding image tags;partitioning the set of input data into a first cluster of data and a second cluster of data based on similarity of the images, wherein the first cluster of data includes a first set of images and corresponding image tags, the first set of images being similar to one another, and wherein the second cluster of data includes a second set of images and corresponding image tags, the second set of images being similar to one another;determining that a size of the first cluster of data exceeds a predefined threshold and a size of the second cluster of data is less than the predefined threshold;based on the size of the first cluster of data exceeding the predefined threshold, partitioning the first set of images and corresponding image tags into a third cluster of data and a fourth cluster of data, wherein the third cluster of data and the fourth cluster data each having a size of data that is less than the predefined threshold; andtraining a classifier that predicts image tags ...

Object Segmentation, Including Sky Segmentation

A digital medium environment includes an image processing application that performs object segmentation on an input image. An improved object segmentation method implemented by the image processing application comprises receiving an input image that includes an object region to be segmented by a segmentation process, processing the input image to provide a first segmentation that defines the object region, and processing the first segmentation to provide a second segmentation that provides pixel-wise label assignments for the object region. In some implementations, the image processing application performs improved sky segmentation on an input image containing a depiction of a sky. 1. In a digital medium environment including an image processing application that performs object segmentation on an input image , an improved object segmentation method implemented by the image processing application , the method comprising:receiving an input image that includes an object region to be segmented; parsing the input image to provide a probability mask which classifies individual pixels in the input image;', 'determining, from a database, multiple images which have layouts at least similar to a layout of the input image, wherein the multiple images include respective masks;', 'processing the respective masks to provide a weighted average mask; and', 'combining the probability mask and the weighted average mask to provide the first segmentation; and, 'processing the input image to provide a first segmentation that defines the object region, said processing comprisingprocessing the first segmentation to provide a second segmentation that provides pixel-wise label assignments for the object region.2. A method as described in claim 1 , wherein the object region to be segmented comprises a depiction of a sky.3. A method as described in claim 1 , wherein parsing the input image to provide the probability mask comprises parsing the input image using a Conditional Random Field to ...

Image Cropping Suggestion Using Multiple Saliency Maps

Image cropping suggestion using multiple saliency maps is described. In one or more implementations, component scores, indicative of visual characteristics established for visually-pleasing croppings, are computed for candidate image croppings using multiple different saliency maps. The visual characteristics on which a candidate image cropping is scored may be indicative of its composition quality, an extent to which it preserves content appearing in the scene, and a simplicity of its boundary. Based on the component scores, the croppings may be ranked with regard to each of the visual characteristics. The rankings may be used to cluster the candidate croppings into groups of similar croppings, such that croppings in a group are different by less than a threshold amount and croppings in different groups are different by at least the threshold amount. Based on the clustering, croppings may then be chosen, e.g., to present them to a user for selection.

ROBUST TRACKING OF OBJECTS IN VIDEOS

The present disclosure is directed toward systems and methods for tracking objects in videos. 1. In a digital environment for processing digital videos , a method of tracking objects in videos comprising:extracting a plurality of video frames from a video;generating an image search index from the plurality of video frames by extracting features from the plurality of video frames;receiving an indication of a query object in one or more key frames of the plurality of video frames;determining similarity scores between the one or more key frames and the plurality of video frames based on a comparison of features of the query object and the extracted features in the image search index; andidentifying the query object in one or more of the video frames based on the similarity scores.2. The method as recited in claim 1 , further comprising:identifying one or more auxiliary key frames; andwherein determining similarity scores further comprising determining similarity scores between the one or more auxiliary key frames and the plurality of video frames.3. The method as recited in claim 2 , wherein identifying one or more auxiliary key frames comprises:selecting a candidate video frame from the image search index;determining a similarity between the candidate video frame and each of the one or more key frames;determining that the similarity between the candidate video frame and a key frame of the one or more key frames is greater than a predetermined threshold; andre-categorizing, based on the similarity being greater than the predetermined threshold, the candidate video frame as an auxiliary key frame.4. The method as recited in claim 3 , further comprising:determining a first candidate query object for the video frame based on the key frame;determining a second candidate query object for the video frame based on the auxiliary key frame;weighting a similarity score for the first candidate query object using a time decay function;weighting a similarity score for the second ...

ACCURATE TAG RELEVANCE PREDICTION FOR IMAGE SEARCH

Embodiments of the present invention provide an automated image tagging system that can predict a set of tags, along with relevance scores, that can be used for keyword-based image retrieval, image tag proposal, and image tag auto-completion based on user input. Initially, during training, a clustering technique is utilized to reduce cluster imbalance in the data that is input into a convolutional neural network (CNN) for training feature data. In embodiments, the clustering technique can also be utilized to compute data point similarity that can be utilized for tag propagation (to tag untagged images). During testing, a diversity based voting framework is utilized to overcome user tagging biases. In some embodiments, bigram re-weighting can down-weight a keyword that is likely to be part of a bigram based on a predicted tag set. 1. A computer-implemented method comprising:inputting an image into a trained image classifier, wherein the trained image classifier is based on a recursive clustering process that generates data clusters, wherein a step in the recursive clustering process comprises partitioning data into clusters and recombining all clusters from the clusters with a size exceeding a predefined threshold prior to a subsequent partitioning;determining, for the image, a related data cluster from the data clusters of the trained image classifier, the related data cluster comprising a set of training images and corresponding tags; andpropagating a tag to the image, the tag selected from the corresponding tags of the set of training images.2. The method of claim 1 , wherein the related data cluster is identified using a similarity analysis of features in the image in relation to the data clusters.3. The method of claim 1 , wherein the tag is selected from the corresponding tags of the set based on a relevance score.4. The method of claim 3 , wherein the relevance score is determined using a confidence value of the tag based on a measure of similarity between two ...

ILLUMINATION ESTIMATION FROM A SINGLE IMAGE

Methods and systems are provided for using a single image of an indoor scene to estimate illumination of an environment that includes the portion captured in the image. A neural network system may be trained to estimate illumination by generating recovery light masks indicating a probability of each pixel within the larger environment being a light source. Additionally, low-frequency RGB images may be generated that indicating low-frequency information for the environment. The neural network system may be trained using training input images that are extracted from known panoramic images. Once trained, the neural network system infers plausible illumination information from a single image to realistically illumination images and objects being manipulated in graphics applications, such as with image compositing, modeling, and reconstruction. 1. A computer-implemented method for training a neural network system to estimate illumination of images , the method comprising:receiving training panoramic images;extracting training patches from the training panoramic images, each training patch being a portion of a training panoramic image;generating training recovery light masks for the training patches using a neural network system, each training recovery light mask indicating a probability of each pixel of a corresponding training panoramic image being a light source;based on comparisons of training light recovery masks to reference masks, training the neural network system to synthesize light recovery masks for input images.2. The computer-implemented method of claim 1 , wherein the reference masks are binary light masks generated from the training panoramic images.3. The computer-implemented method of claim 2 , wherein the binary light masks are generated by one or more linear support vector machine classifiers trained to classify each pixel within the training panoramic images as a light source or not a light source.4. The computer-implemented method of claim 3 , wherein ...

TOPIC ASSOCIATION AND TAGGING FOR DENSE IMAGES

A framework is provided for associating dense images with topics. The framework is trained utilizing images, each having multiple regions, multiple visual characteristics and multiple keyword tags associated therewith. For each region of each image, visual features are computed from the visual characteristics utilizing a convolutional neural network, and an image feature vector is generated from the visual features. The keyword tags are utilized to generate a weighted word vector for each image by calculating a weighted average of word vector representations representing keyword tags associated with the image. The image feature vector and the weighted word vector are aligned in a common embedding space and a heat map is computed for the image. Once trained, the framework can be utilized to automatically tag images and rank the relevance of images with respect to queried keywords based upon associated heat maps. 1. A computer system comprising:one or more processors; andone or more computer storage media storing computer-useable instructions that, when used by the one or more processors, cause the one or more processors to:receive a plurality of images, each image of the plurality of images being associated with a plurality of tags and each image of the plurality of images being comprised of a plurality of regions, each region of each image comprising less than an entirety of the image it comprises;for each region of each image of the plurality of images, generate an image feature vector from one or more visual features;for each image of the plurality of images, generate a weighted word vector from the associated plurality of tags; andfor each image, compute a heat map corresponding thereto by aligning the image feature vector for each region of a given image and the weighted word feature vector into a common embedding space utilizing cosine similarity loss, wherein a plurality of regions of the heat map corresponds to the plurality of regions of the given image and ...

LARGE-SCALE IMAGE TAGGING USING IMAGE-TO-TOPIC EMBEDDING

A framework is provided for associating images with topics utilizing embedding learning. The framework is trained utilizing images, each having multiple visual characteristics and multiple keyword tags associated therewith. Visual features are computed from the visual characteristics utilizing a convolutional neural network and an image feature vector is generated therefrom. The keyword tags are utilized to generate a weighted word vector (or “soft topic feature vector”) for each image by calculating a weighted average of word vector representations that represent the keyword tags associated with the image. The image feature vector and the soft topic feature vector are aligned in a common embedding space and a relevancy score is computed for each of the keyword tags. Once trained, the framework can automatically tag images and a text-based search engine can rank image relevance with respect to queried keywords based upon predicted relevancy scores. 1. A computer system comprising:one or more processors; and receive a plurality of images, each image of the plurality of images being associated with a plurality of tags; and', generate a weighted word vector from the associated plurality of tags;', 'generate an image feature vector from one or more visual features associated with the subject image;', 'align the image feature vector and the weighted word vector in a common embedding space; and', 'using the aligned vectors, compute a relevancy score for each of the associated plurality of tags as it pertains to the subject image., 'for each subject image of the plurality of images], 'one or more computer storage media storing computer-usable instructions that, when used by the one or more processors, cause the one or more processors to2. The computing system of claim 1 , wherein for each image of the plurality of images claim 1 , the one or more processors are further caused to compute the one or more visual features.3. The computing system of claim 2 , wherein the one or ...

Digital Image Defect Identification and Correction

Digital image defect identification and correction techniques are described. In one example, a digital medium environment is configured to identify and correct a digital image defect through identification of a defect in a digital image using machine learning. The identification includes generating a plurality of defect type scores using a plurality of defect type identification models, as part of machine learning, for a plurality of different defect types and determining the digital image includes the defect based on the generated plurality of defect type scores. A correction is generated for the identified defect and the digital image is output as included the generated correction.

UTILIZING DEEP LEARNING TO RATE ATTRIBUTES OF DIGITAL IMAGES

Systems and methods are disclosed for estimating aesthetic quality of digital images using deep learning. In particular, the disclosed systems and methods describe training a neural network to generate an aesthetic quality score digital images. In particular, the neural network includes a training structure that compares relative rankings of pairs of training images to accurately predict a relative ranking of a digital image. Additionally, in training the neural network, an image rating system can utilize content-aware and user-aware sampling techniques to identify pairs of training images that have similar content and/or that have been rated by the same or different users. Using content-aware and user-aware sampling techniques, the neural network can be trained to accurately predict aesthetic quality ratings that reflect subjective opinions of most users as well as provide aesthetic scores for digital images that represent the wide spectrum of aesthetic preferences of various users. 1. A computer-implemented method of estimating aesthetic quality of digital images using deep learning , the method comprising:receiving a digital image;generating a set of features for the digital image using a set of convolutional layers of a neural network;generating an attribute quality score for each of a plurality of attributes by processing the set of features generated using the set of convolutional layers of the neural network using a plurality of individual attribute models, the individual attribute models comprising individual sets of fully-connected layers in the neural network; andgenerating an aesthetic quality score for the digital image by processing the set of features generated using the set of convolutional layers of the neural network using an aesthetic quality model, the aesthetic quality model comprising a set of fully-connected layers in the neural network trained using a regression loss model.2. The method as recited in claim 1 , wherein the plurality of ...

AUTOMATICALLY SEGMENTING IMAGES BASED ON NATURAL LANGUAGE PHRASES

The invention is directed towards segmenting images based on natural language phrases. An image and an n-gram, including a sequence of tokens, are received. An encoding of image features and a sequence of token vectors are generated. A fully convolutional neural network identifies and encodes the image features. A word embedding model generates the token vectors. A recurrent neural network (RNN) iteratively updates a segmentation map based on combinations of the image feature encoding and the token vectors. The segmentation map identifies which pixels are included in an image region referenced by the n-gram. A segmented image is generated based on the segmentation map. The RNN may be a convolutional multimodal RNN. A separate RNN, such as a long short-term memory network, may iteratively update an encoding of semantic features based on the order of tokens. The first RNN may update the segmentation map based on the semantic feature encoding. 1. A computer-readable storage medium having instructions stored thereon for segmenting an image that includes a plurality of pixels , which , when executed by a processor of a computing device cause the computing device to perform actions comprising:receiving an ordered set of tokens that references a first region of the image;generating an image map that represents a correspondence between each of a plurality of image features and a corresponding portion of the plurality of pixels;generating a set of token data elements, wherein each of the token data elements represents semantic features of a corresponding token of the set of tokens;iteratively updating a segmentation map that represents whether each of the plurality of pixels is included in the first region of the image, wherein each of a plurality of iterative updates of the segmentation map is based on a previous version of the segmentation map and a combination of the image map and one of the token data elements that is based on an order of the set of tokens; andgenerating ...

PLANAR REGION GUIDED 3D GEOMETRY ESTIMATION FROM A SINGLE IMAGE

Techniques for planar region-guided estimates of 3D geometry of objects depicted in a single 2D image. The techniques estimate regions of an image that are part of planar regions (i.e., flat surfaces) and use those planar region estimates to estimate the 3D geometry of the objects in the image. The planar regions and resulting 3D geometry are estimated using only a single 2D image of the objects. Training data from images of other objects is used to train a CNN with a model that is then used to make planar region estimates using a single 2D image. The planar region estimates, in one example, are based on estimates of planarity (surface plane information) and estimates of edges (depth discontinuities and edges between surface planes) that are estimated using models trained using images of other scenes. 1. A method , performed by a computing device , for enhancing an image based on planar-region-guided estimates of 3D geometry of objects depicted in the image , the method comprising:determining planarity and edge strength of pixels of the image, wherein the determining of planarity and edge strength of the pixels of the image is based on the image of the objects and is not based on additional images of the objects;determining whether pixels of the image are within common planar regions based on the determining of planarity and edge strength of the pixels of the image;determining 3D geometry values of pixels in the common planar regions based on a planar region constraint that requires a relationship between the 3D geometry values of pixels within common planar regions; andenhancing the image by using the 3D geometry values of the pixels to provide the 3D geometry of the objects in the image.2. The method of claim 1 , wherein determining the 3D geometry values of the pixels in common planar regions comprises selecting normals of the pixels of the image based on the planar region constraint requiring similar normals of pixels in common planar regions.3. The method of ...

Sky editing based on image composition

Embodiments of the present disclosure relate to a sky editing system and related processes for sky editing. The sky editing system includes a composition detector to determine the composition of a target image. A sky search engine in the sky editing system is configured to find a reference image with similar composition with the target image. Subsequently, a sky editor replaces content of the sky in the target image with content of the sky in the reference image. As such, the sky editing system transforms the target image into a new image with a preferred sky background.

Utilizing deep learning for rating aesthetics of digital images

Systems and methods are disclosed for estimating aesthetic quality of digital images using deep learning. In particular, the disclosed systems and methods describe training a neural network to generate an aesthetic quality score digital images. In particular, the neural network includes a training structure that compares relative rankings of pairs of training images to accurately predict a relative ranking of a digital image. Additionally, in training the neural network, an image rating system can utilize content-aware and user-aware sampling techniques to identify pairs of training images that have similar content and/or that have been rated by the same or different users. Using content-aware and user-aware sampling techniques, the neural network can be trained to accurately predict aesthetic quality ratings that reflect subjective opinions of most users as well as provide aesthetic scores for digital images that represent the wide spectrum of aesthetic preferences of various users.

USER-GUIDED IMAGE COMPLETION WITH IMAGE COMPLETION NEURAL NETWORKS

Certain embodiments involve using an image completion neural network to perform user-guided image completion. For example, an image editing application accesses an input image having a completion region to be replaced with new image content. The image editing application also receives a guidance input that is applied to a portion of a completion region. The image editing application provides the input image and the guidance input to an image completion neural network that is trained to perform image-completion operations using guidance input. The image editing application produces a modified image by replacing the completion region of the input image with the new image content generated with the image completion network. The image editing application outputs the modified image having the new image content. 1. A method in which one or more processing devices perform operations comprising:accessing an input image having a completion region to be replaced with new image content;receiving a guidance input that is applied to a portion of the completion region; providing the input image and the guidance input to an image completion neural network, wherein the image completion neural network is trained to generate new image content as a function of the guidance input and the input image,', 'generating, with the image completion neural network, the new image content as a function of the guidance input and the input image, and', 'replacing the completion region of the input image with the new image content to produce a modified image; and, 'transforming the input image into a modified image, wherein transforming the input image into the modified image comprisesoutputting the modified image.2. The method of claim 1 , wherein generating the new image content comprises:matching, via the image completion neural network, a portion of the input image having one or more visual attributes corresponding to the guidance input; andapplying an image-completion operation to the portion ...

Object Detection Using Cascaded Convolutional Neural Networks

Different candidate windows in an image are identified, such as by sliding a rectangular or other geometric shape of different sizes over an image to identify portions of the image (groups of pixels in the image). The candidate windows are analyzed by a set of convolutional neural networks, which are cascaded so that the input of one convolutional neural network layer is based on the input of another convolutional neural network layer. Each convolutional neural network layer drops or rejects one or more candidate windows that the convolutional neural network layer determines does not include an object (e.g., a face). The candidate windows that are identified as including an object (e.g., a face) are analyzed by another one of the convolutional neural network layers. The candidate windows identified by the last of the convolutional neural network layers are the indications of the objects (e.g., faces) in the image. 1. A method comprising:identifying multiple candidate windows in an image, each candidate window including a group of pixels of the image, the multiple candidate windows including overlapping candidate windows;identifying one or more of the multiple candidate windows that include an object, the identifying including analyzing the multiple candidate windows using cascaded convolutional neural networks, the cascaded convolutional neural networks including multiple cascade layers, each cascade layer comprising a convolutional neural network, the multiple cascade layers including a first cascade layer that analyzes the identified multiple candidate windows, a second cascade layer that analyzes ones of the multiple candidate windows identified by the first cascade layer as including an object, and a third cascade layer that analyzes ones of the multiple candidate windows identified by the second cascade layer as including an object; andoutputting, as an indication of one or more objects in the image, an indication of one or more of the multiple candidate ...

BLIND-MATE INTEGRATED CONNECTOR

The present invention provides a blind-mate integrated connector, including: a first installation plate, a mechanical part, and a second installation plate; a first guiding structure and first connection ends of at least two sub-connectors are installed in the mechanical part; the first installation plate is connected to the mechanical part; the second installation plate is disposed with second connection ends matching the first connection ends of the sub-connectors in the mechanical part, and the second installation plate is further disposed with a second guiding structure matching the first guiding structures. By practicing the present invention, multiple sub-connectors may be flexibly integrated without a need to design a dedicated connector mold, thereby achieving cost savings and shortening a development cycle.

IDENTIFYING UNKNOWN PERSON INSTANCES IN IMAGES

Methods and systems for recognizing people in images with increased accuracy are disclosed. In particular, the methods and systems divide images into a plurality of clusters based on common characteristics of the images. The methods and systems also determine an image cluster to which an image with an unknown person instance most corresponds. One or more embodiments determine a probability that the unknown person instance is each known person instance in the image cluster using a trained cluster classifier of the image cluster. Optionally, the methods and systems determine context weights for each combination of an unknown person instance and each known person instance using a conditional random field algorithm based on a plurality of context cues associated with the unknown person instance and the known person instances. The methods and systems calculate a contextual probability based on the cluster-based probabilities and context weights to identify the unknown person instance. 1. A method of identifying people in digital images using cluster-based person recognition comprising:dividing, by at least one processor, images of an image gallery into a plurality of image clusters, each image cluster comprising a plurality of images from the image gallery that share one or more common characteristics, the one or more characteristics comprising one or more characteristics other than identities of the plurality of person instances in the images;training, by the at least one processor, a cluster classifier for each image cluster of the plurality of image clusters based on a plurality of known person instances;determining, by the at least one processor, an image cluster of the plurality of image clusters to which an image comprising an unknown person instance most corresponds based on one or more characteristics of the image and common characteristics of the determined image cluster of the plurality of image clusters, the one or more characteristics of the image comprising ...

Joint Depth Estimation and Semantic Segmentation from a Single Image

Joint depth estimation and semantic labeling techniques usable for processing of a single image are described. In one or more implementations, global semantic and depth layouts are estimated of a scene of the image through machine learning by the one or more computing devices. Local semantic and depth layouts are also estimated for respective ones of a plurality of segments of the scene of the image through machine learning by the one or more computing devices. The estimated global semantic and depth layouts are merged with the local semantic and depth layouts by the one or more computing devices to semantically label and assign a depth value to individual pixels in the image. 1. A method of performing joint depth estimation and semantic labeling of an image by one or more computing devices , the method comprising:estimating global semantic and depth layouts of a scene of the image through machine learning by the one or more computing devices;estimating local semantic and depth layouts for respective ones of a plurality of segments of the scene of the image through machine learning by the one or more computing devices; andmerging the estimated global semantic and depth layouts with the estimated local semantic and depth layouts by the one or more computing devices to semantically label and assign a depth value to individual pixels in the image.2. A method as described in claim 1 , wherein the estimating of the global semantic and depth layouts is performed as a template classification problem by selecting one or more of a plurality of global templates having corresponding global semantic and depth layouts as corresponding to the scene of the image.3. A method as described in claim 2 , wherein the selecting is performed using a plurality of the global templates in combination to perform the estimating of the global semantic and depth layout of the scene of the image.4. A method as described in claim 2 , further comprising generating the plurality of global templates ...

GENERATING IMAGE FEATURES BASED ON ROBUST FEATURE-LEARNING

Techniques for increasing robustness of a convolutional neural network based on training that uses multiple datasets and multiple tasks are described. For example, a computer system trains the convolutional neural network across multiple datasets and multiple tasks. The convolutional neural network is configured for learning features from images and accordingly generating feature vectors. By using multiple datasets and multiple tasks, the robustness of the convolutional neural network is increased. A feature vector of an image is used to apply an image-related operation to the image. For example, the image is classified, indexed, or objects in the image are tagged based on the feature vector. Because the robustness is increased, the accuracy of the generating feature vectors is also increased. Hence, the overall quality of an image service is enhanced, where the image service relies on the image-related operation. 1. A computer-implemented method for increasing robustness of a convolutional neural network that supports at least one of image classification , image tagging , or image retrieval , the computer-implemented method comprising:accessing, by a computer system, a first training dataset comprising first image data, the first training dataset associated with a first image-related task and a first label applicable to the first image data;accessing, by the computer system, a second training dataset comprising second training data, the second training dataset associated with a second task and a second label; minimizing a first loss function for the first training dataset based on the first image-related task and a second loss function for the second training dataset based on the second task, and', 'updating parameters of the convolutional neural network based on the minimizing of the first loss function and the second loss function;, 'training, by the computer system, the convolutional neural network by at leastinputting, by the computer system, image data of an ...

SEMANTIC CLASS LOCALIZATION IN IMAGES

Semantic class localization techniques and systems are described. In one or more implementation, a technique is employed to back communicate relevancies of aggregations back through layers of a neural network. Through use of these relevancies, activation relevancy maps are created that describe relevancy of portions of the image to the classification of the image as corresponding to a semantic class. In this way, the semantic class is localized to portions of the image. This may be performed through communication of positive and not negative relevancies, use of contrastive attention maps to different between semantic classes and even within a same semantic class through use of a self-contrastive technique. 1. In a digital medium classification environment , a method implemented by at least one computing device , the method comprising:aggregating, by the at least one computing device, patterns of neurons in a neural network by progressing through a sequence of layers of the neural network to classify an image as relating to a semantic class;communicating, by the at least one computing device, positive relevancy of the patterns formed by the neurons to the semantic class by progressing backwards through the sequence of layers of the neural network, wherein the communicating of the positive relevancy of the pattern between a plurality of layers from the sequence of layers is based on a probabilistic Winner-Take-All (WTA) approach;localizing, by the at least one computing device, the semantic class within the image based on the communicated positive relevancy of the aggregated patterns to the semantic class; andgenerating, by the at least one computing device, digital content based on localization of the semantic class within the image.2. The method as described in claim 1 , wherein the semantic class identifies an object included in the image or emotional feeling expressed in the image.3. (canceled)4. The method as described in claim 1 , wherein the communicating of ...

RECURRENT NEURAL NETWORK ARCHITECTURES WHICH PROVIDE TEXT DESCRIBING IMAGES

Provided are systems and techniques that provide an output phrase describing an image. An example method includes creating, with a convolutional neural network, feature maps describing image features in locations in the image. The method also includes providing a skeletal phrase for the image by processing the feature maps with a first long short-term memory (LSTM) neural network trained based on a first set of ground truth phrases which exclude attribute words. Then, attribute words are provided by processing the skeletal phrase and the feature maps with a second LSTM neural network trained based on a second set of ground truth phrases including words for attributes. Then, the method combines the skeletal phrase and the attribute words to form the output phrase.

TRAINING AND UTILIZING AN IMAGE EXPOSURE TRANSFORMATION NEURAL NETWORK TO GENERATE A LONG-EXPOSURE IMAGE FROM A SINGLE SHORT-EXPOSURE IMAGE

The present disclosure relates to training and utilizing an image exposure transformation network to generate a long-exposure image from a single short-exposure image (e.g., still image). In various embodiments, the image exposure transformation network is trained using adversarial learning, long-exposure ground truth images, and a multi-term loss function. In some embodiments, the image exposure transformation network includes an optical flow prediction network and/or an appearance guided attention network. Trained embodiments of the image exposure transformation network generate realistic long-exposure images from single short-exposure images without additional information.

Remote Radio Apparatus And Component Thereof

An example remote radio apparatus is provided, including a body, a mainboard, a mainboard heat sink, a maintenance cavity, an optical module, and an optical module heat sink. The maintenance cavity and the optical module heat sink are integrally connected, while the optical module is mounted on a bottom surface of the optical module heat sink. The maintenance cavity and the optical module heat sink are mounted on a side surface of the body, and the mainboard heat sink is mounted on and covers the mainboard. The mainboard heat sink and the mainboard are installed on a front surface of the body, and the mainboard heat sink and the optical module heat sink are spaced by a preset distance. The temperature of the optical module is controlled within a range required by a specification. 1the maintenance cavity and the optical module heat sink are integrally connected;the optical module is mounted on a bottom surface of the optical module heat sink;the maintenance cavity and the optical module heat sink are mounted on a side surface of the body; andthe mainboard heat sink is mounted on and covers the mainboard, wherein the mainboard heat sink and the mainboard are mounted on a front surface of the body, and wherein the mainboard heat sink and the optical module heat sink are spaced by a preset distance.. A remote radio apparatus (RRU), comprising: a body, a mainboard, a mainboard heat sink, a maintenance cavity, an optical module, and an optical module heat sink, wherein This application is a continuation of U.S. patent application Ser. No. 16/002,608, filed on Jun. 7, 2018, which is a continuation of International Patent Application No. PCT/CN2016/108867, filed on Dec. 7, 2016, which claims priority to Chinese Patent Application No. 201510896866.X, filed on Dec. 8, 2015, and Chinese Patent Application No. 201521010074.X, filed on Dec. 8, 2015. All of the aforementioned patent applications are hereby incorporated by reference in their entireties.The present application ...

Event Image Curation

In embodiments of event image curation, a computing device includes memory that stores a collection of digital images associated with a type of event, such as a digital photo album of digital photos associated with the event, or a video of image frames and the video is associated with the event. A curation application implements a convolutional neural network, which receives the digital images and a designation of the type of event. The convolutional neural network can then determine an importance rating of each digital image within the collection of the digital images based on the type of the event. The importance rating of a digital image is representative of an importance of the digital image to a person in context of the type of the event. The convolutional neural network generates an output of representative digital images from the collection based on the importance rating of each digital image. 1. A method for event image curation , the method comprising:receiving a collection of digital images as an input to a convolutional neural network, the digital images being associated with a type of event;determining, using the convolutional neural network, an importance rating of each digital image within the collection of the digital images based on the type of the event, the importance rating of a digital image representative of an importance of the digital image to a person in context of the type of the event; andgenerating an output of representative digital images from the collection based on the importance rating of each digital image.2. The method as recited in claim 1 , wherein:the collection of the digital images is a digital photo album of digital photos that are associated with the type of the event; andthe representative digital images are a set of the digital photos that are representative of important moments during the event.3. The method as recited in claim 2 , further comprising:determining a diversity of the set of the digital photos to identify one ...

Convolutional Neural Network Joint Training

In embodiments of convolutional neural network joint training, a computing system memory maintains different data batches of multiple digital image items, where the digital image items of the different data batches have some common features. A convolutional neural network (CNN) receives input of the digital image items of the different data batches, and classifier layers of the CNN are trained to recognize the common features in the digital image items of the different data batches. The recognized common features are input to fully-connected layers of the CNN that distinguish between the recognized common features of the digital image items of the different data batches. A scoring difference is determined between item pairs of the digital image items in a particular one of the different data batches. A piecewise ranking loss algorithm maintains the scoring difference between the item pairs, and the scoring difference is used to train CNN regression functions. 1. A computing system implemented for convolutional neural network joint training , the system comprising:memory configured to maintain different data batches each including multiple digital image items, the multiple digital image items of the different data batches having at least some common features; receive input of the multiple digital image items of the different data batches;', 'train classifier layers of the convolutional neural network to recognize the common features in the multiple digital image items of the different data batches;', 'input the recognized common features to fully-connected layers of the convolutional neural network, each of the fully-connected layers corresponding to a different one of the different data batches, the fully-connected layers configured to distinguish between the recognized common features of the multiple digital image items of the different data batches;', 'determine a scoring difference between item pairs of the multiple digital image items in a particular one of the ...

Accelerating Object Detection

Accelerating object detection techniques are described. In one or more implementations, adaptive sampling techniques are used to extract features from an image. Coarse features are extracted from the image and used to generate an object probability map. Then, dense features are extracted from high-probability object regions of the image identified in the object probability map to enable detection of an object in the image. In one or more implementations, cascade object detection techniques are used to detect an object in an image. In a first stage, exemplars in a first subset of exemplars are applied to features extracted from the multiple regions of the image to detect object candidate regions. Then, in one or more validation stages, the object candidate regions are validated by applying exemplars from the first subset of exemplars and one or more additional subsets of exemplars. 1. A computer-implemented method comprising:selecting exemplars based on an image, the exemplars taken from example images as examples of an object that is detectable in each example image;grouping the selected exemplars into at least a first subset of exemplars and a second subset of exemplars;extracting features from multiple regions of the image;applying each exemplar in the first subset of exemplars to the features extracted from the multiple regions of the image to detect object candidate regions;extracting additional features from the object candidate regions; andapplying each exemplar in the first subset of exemplars and the second subset of exemplars to the features and the additional features extracted from the object candidate regions to validate one or more of the object candidate regions.2. The computer-implemented method of claim 1 , wherein the number of exemplars in the first subset of exemplars is less than the number of exemplars in the second subset of exemplars.3. The computer-implemented method of claim 1 , wherein the applying each exemplar in the first subset of ...

Compositing Aware Digital Image Search

Compositing aware digital image search techniques and systems are described that leverage machine learning. In one example, a compositing aware image search system employs a two-stream convolutional neural network (CNN) to jointly learn feature embeddings from foreground digital images that capture a foreground object and background digital images that capture a background scene. In order to train models of the convolutional neural networks, triplets of training digital images are used. Each triplet may include a positive foreground digital image and a positive background digital image taken from the same digital image. The triplet also contains a negative foreground or background digital image that is dissimilar to the positive foreground or background digital image that is also included as part of the triplet. 1. In a digital medium compositing aware digital image search environment , a method implemented by at least one computing device , the method comprising:receiving, by the at least one computing device, foreground features extracted from a foreground of a digital image using machine learning; background features extracted from backgrounds of the plurality of candidate digital images, respectively, using machine learning; and', 'the foreground features from the foreground of the digital image;, 'searching, by the at least one computing device, a plurality of candidate digital images based ongenerating, by the at least one computing device, a search result including at least one of the plurality of candidate digital images based on the searching; andoutputting, by the at least one computing device, the search result.2. The method as described in claim 1 , wherein the background features and the foreground features are extracted using a two-stream neural network.3. The method as described in claim 1 , wherein the searching includes calculating scores through feature embedding based on the foreground features from the digital image and the background features ...

FACILITATING PRESERVATION OF REGIONS OF INTEREST IN AUTOMATIC IMAGE CROPPING

Embodiments of the present invention are directed to facilitating region of interest preservation. In accordance with some embodiments of the present invention, a region of interest preservation score using adaptive margins is determined. The region of interest preservation score indicates an extent to which at least one region of interest is preserved in a candidate image crop associated with an image. A region of interest positioning score is determined that indicates an extent to which a position of the at least one region of interest is preserved in the candidate image crop associated with the image. The region of interest preservation score and/or the preserving score are used to select a set of one or more candidate image crops as image crop suggestions. 1. A computer system comprising:a processor; anda computer storage medium storing computer-useable instructions that, when used by the processor, cause the processor to:identifying a first relative position of an aggregate region in relation to the image, the aggregate region including at least one region of interest in the image;identifying a second relative position of the aggregate region in relation to a candidate image crop associated with an image;determining an extent to which the aggregate region in the image maintains a relative position in the candidate image crop based on a comparison of the first relative position and the second relative position; andgenerating a candidate score for the candidate image crop utilizing the extent to which the aggregate region maintains the relative position in the image in the candidate image crop, the candidate score being used to automatically select the candidate image crop and provide the candidate image crop as an image crop suggestion for automated image cropping.2. The computer system of further comprising:identifying the at least one region of interest in the image; andusing the at least one region of interest to determine the aggregate region.3. The computer ...