ПРЕОБРАЗОВАНИЕ ПОТОКОВ СОБЫТИЙ

Accessing databases

A method of accessing multiple independent databases (e.g. retail and financial) with a single query having multiple expressions, the method comprises: deriving from a single query at least one filtering query containing at least one filtering expression and a target query containing at least one target expression; searching a first one of the multiple independent databases using the filtering query to obtain a filtering set of target entries matching the filtering expression; applying identifiers only of the filtering set of target entries and the target query to a second one of the multiple independent databases to search amongst the following set of target entries only in the second database for entries that match the target expression; and generating a set of result entries from the second database which thereby satisfy the filtering expression and the target expression. Data joining may be employed to join data belonging to the same entity, external data owned by a plurality of entities ...

Building datasets in data processing pipelines

Database management program and database management device

Resource management systems and methods

Example resource management systems and methods are described. In one implementation, a resource manager is configured to manage data processing tasks associated with multiple data elements. An execution platform is coupled to the resource manager and includes multiple execution nodes configured to store data retrieved from multiple remote storage devices. Each execution node includes a cache and a processor, where the cache and processor are independent of the remote storage devices. A metadata manager is configured to access metadata associated with at least a portion of the multiple data elements.

GENERATING DATA VISUALIZATIONS ACCORDING TO AN OBJECT MODEL OF SELECTED DATA SOURCES

The method receives a visual specification, which specifies a data source, visual variables, and data fields from the data source. Each visual variable is associated with data fields and each data field is either a dimension or a measure. From an object model of the data source, the method identifies a minimal subtree that includes all of the dimension data fields and constructs a query from the minimal subtree. The method executes the query against the data source to retrieve a set of tuples, each tuple comprising a unique ordered combination of data values for the dimension data fields. For each tuple, the method forms an extended tuple by appending aggregated data values corresponding to each measure data field. The method then builds and displays a data visualization according to the data fields in the extended tuples and according to the visual variables to which the data fields are associated.

CHEMICAL SUBSTRUCTURE SEARCH QUERY BUILDING AND EXECUTION

Systems and methods for enabling construction of a complex Boolean chemical substructure query in a structured graphical user interface are provided. The chemical substructures (molecules) may be represented graphically in standard molecular notation, and may be arranged horizontally and vertically on the interface, along with Boolean logical operators. Boolean logical operators of a first type may logically associate molecules arranged in horizontal fashion to form row queries, whereas Boolean logical operators of a different, second type may logically associate the row queries to form a composite query to be applied to a database of molecules. The operators of the first type may comprise disjunctive operators, whereas the operators of the second type may comprise conjunctive operators.

TIME-SERIES PATTERN MATCHING SYSTEM

A system includes a pattern engine that, in response to selection by a user of a first data set, generates a similarity self-join of the first data set for a specified length. The similarity self-join indicates, for each reference subsequence, a minimum value of distances between the reference subsequence and other subsequences within the first data set. A user portal generates a user interface visually representing the first data set and identifying two subsequences that correspond to the lowest value of the similarity self-join. An alert system receives an alert request specifying a second data set and a pattern specification. An incremental pattern engine generates an initial state of a similarity join of the second data set and the pattern specification. In response to data being received for the first data set, the alert system transmits an alert message if any value of the similarity join meets a threshold.

DATA ANALYSIS SYSTEM

A data analysis and display system and method for performing a plurality of operations on data arranged in data tables. One or more of the following data operations are provided: combining separate data together, for example by joining and appending; normalizing and summarizing; copying; calculating and presenting new information from existing data; making corrections in a persistent manner, and analyzing the data, to allow users to work with their data at a high level using a core set of functions that facilitate performance of these operations in a generally simple, intuitive manner.

DATA ANALYSIS SYSTEM

A data analysis and display system and method for performing a plurality of operations on data arranged in data tables. One or more of the following data operations are provided: combining separate data together, for example by joining and appending; normalizing an d summarizing; copying; calculating and presenting new information from existing data; maki ng corrections in a persistent manner, and analyzing the data, to allow users to work with the ir data at a high level using a core set of functions that facilitate performance of these operations in a generally simple, intuitive manner.

DATA ANALYSIS SYSTEM

A data analysis and display system and method for performing a plurality of operations on data arranged in data tables. One or more of the following data operations are provided: combining separate data together, for example by joining and appending; normalizing and summarizing; copying; calculating and presenting new information from existing data; making corrections in a persistent maimer, and analyzing the data, to allow users to work with their data at a high level using a core set of functions that facilitate performance of these operations in a generally simple, intuitive maimer.

QUERYING JOINED DATA WITHIN A SEARCH ENGINE INDEX

Techniques and systems for indexing and retrieving data and documents stored in a record-based database management system (RDBMS) utilize a search engine interface. Search-engine indices are created from tables in the RDBMS and data from the tables is used to create "documents" for each record. Queries that require data from multiple tables may be parsed into a primary query and a set of one or more secondary queries. Join mappings and documents are created for the necessary tables. Documents matching the query string are retrieved using the search-engine indices and join mappings.

METHOD AND APPARATUS FOR PERFORMING A JOIN QUERY IN A DATABASE SYSTEM

A computer implemented method for generating a response to a join. Two tables are divided into fragments. A join fragment map identifies, for each respective fragment from the first table, the fragments from the second table which contain a record that satisfies the join predicate with a record from the respective fragment from the first table. The map is used to eliminate fragments which cannot satisfy the join predicate.

AGGREGATING AND UPDATING HETEROGENEOUS DATA OBJECTS

Systems and methods for data processing

A computer-implemented method is provided for permitting a user to manipulate data feeds via a graphical user interface. The method includes receiving, by a computing device, the data feeds over a communications network in real time from corresponding data sources and receiving from the user, by the computing device, via the graphical user interface an execution plan comprising a list of one or more actions to be performed on the data feeds. The method also includes sequentially executing, by the computing device, each of the actions in the execution plan on the real-time data feeds to generate one or more manipulated data feeds. The method further includes displaying, by the computing device, the manipulated data feeds to the user in a user-defined format via the graphical user interface.

RESOURCE MANAGEMENT SYSTEMS AND METHODS

Example resource management systems and methods are described. In one implementation, a processing device is to: provide a plurality of virtual processors, each virtual processor comprising a cache memory to cache a portion of database data accessed from a storage platform; receive a query referencing the database data stored in the storage platform; determine at least one task to process the received query; and allocate the at least one task to a subset of the plurality of virtual processors to process the at least one task. The allocation may be based at least in part on the respective portions of the database data that are cached within the respective cache memories of the subset of the plurality of virtual processors.

Dynamic modeling of data in relational databases

The disclosed embodiments provide a system that facilitates making updates to a data model, wherein the data model is associated with data items stored in a relational database system. In particular, updates to the data model are automatically propagated to the relational database system so that the structure of the relational database system remains consistent with the data model. This is accomplished by automatically executing queries corresponding to the updates against the relational database system. For example, changes to fields and/or tables in the data model can be automatically propagated to a relational database system by executing “Create,” “Drop,” “Alter,” and/or “Add” statements against corresponding fields and/or tables in the relational database system.

Implementation of semi-structured data as a first-class database element

A system, apparatus, and method for managing data storage and data access for semi-structured data systems.

GPU enabled database systems

Methods for resolving a number of in-memory issues associated with parallel query execution of a database operation on a database utilizing a graphics processing unit (GPU) are presented including: tying a table choice to a number of accesses per second made to a table; and synchronizing threads in a same shared GPU multiprocessor to avoid compromising concurrency, and where the parallel query execution of the database operation is performed solely by the GPU. In some embodiments, methods further include storing data from the GPU to a disk to solve volatility; and enabling a user, at any time, to query the amount of memory being used by the table created by the user to monitor memory consumption.

Adaptive distribution method for hash operation

A method, apparatus, and system for join operations of a plurality of relations that are distributed over a plurality of storage locations over a network of computing components.

Estimating the size of a join by generating and combining partial join estimates

A program product, an apparatus, and a method of estimating a join size by generating partial join estimates and combining the partial join estimates. In particular, robust formulas are provided that may result in more consistent estimates and fewer inaccuracies. Inconsistencies and inaccuracies are generally present in conventional join size estimation techniques, however, consistent with the invention, consistent join size estimates may be generated resulting in improved access plan selection during optimization.

Hybrid query execution plan

A procedural pattern in a received query execution plan can be matched to a stored pattern for which an equivalent declarative operator has been pre-defined. The query execution plan can describe a query for accessing data. A hybrid execution plan can be generated by replacing the procedural pattern with the equivalent declarative operator. A hybrid execution plan processing cost can be assigned to execution of the hybrid execution plan and a query execution plan processing cost can be assigned to execution of the query execution plan. The assigning can include evaluating a cost model for the hybrid execution plan and the query execution plan. The query can be executed using the hybrid execution plan if the hybrid execution plan processing cost is less than the query execution plan processing cost or the query execution plan if the hybrid execution plan processing cost is greater than the query execution plan processing cost. Related systems, methods, and articles of manufacture are disclosed ...

PARTITIONED JOIN WITH DENSE INNER TABLE REPRESENTATION

To perform a join operation on database objects, data structures contained in a first database object are distributed across database partitions in accordance with a partitioning scheme. Data structures of the first database object are associated with respective indices computed complementarily to the partitioning scheme. Other indices are computed from the respective data structures of a second database object. The join operation is performed at each of the database partitions on the data structures in the respective first and second database objects having the indices and the other indices in common.

TECHNIQUES FOR TIME INTERSECTION

Techniques to intersect time information include receiving a set of time tokens, the set of time tokens comprising at least a first time token and a second time token; performing a compatibility check of joining the first time token and the second time token into a combined time token; and performing a join attempt of joining the first time token and the second time token into the combined time token where the compatibility check is positive. Other embodiments are described and claimed.

DATABASE PARTITION PRUNING USING DEPENDENCY GRAPH

Provided is a system and method for pruning partitions from a database access operation based on a dependency graph. In one example, the method may include generating a dependency graph for a partition-wise operation, the dependency graph comprising nodes representing partition candidates and links between the nodes identifying dependencies of the partition candidates, receiving, at runtime, a database query comprising a partition identifier, identifying a partition candidate that can be excluded from processing the database query based on the partition identifier, pruning a second partition candidate based on a dependency in the dependency graph between the excluded partition candidate and the second partition candidate, and performing a database access for the database query based on the pruning.

ESTIMATING A COST OF PERFORMING DATABASE OPERATIONS USING VECTORIZED INSTRUCTIONS

Techniques for performing database operations using vectorized instructions are provided. In one technique, it is determined whether to perform a database operation using one or more vectorized instructions or without using any vectorized instructions. This determination may comprise estimating a first cost of performing the database operation using one or more vectorized instructions and estimating a second cost of performing the database operation without using any vectorized instructions. Multiple factors that may be used to determine which approach to follow, such as the number of data elements that may fit into a SIMD register, a number of vectorized instructions in the vectorized approach, a number of data movement instructions that involve moving data from a SIMD register to a non-SIMD register and/or vice versa, a size of a cache, and a projected size of a hash table.

HASH JOIN USING COLLABORATIVE PARALLEL FILTERING IN INTELLIGENT STORAGE WITH OFFLOADED BLOOM FILTERS

Processing resources at a storage system for a database server are utilized to perform aspects of a join operation that would conventionally be performed by the database server. When requesting a range of data units from a storage system, the database server includes join metadata describing aspects of the join operation for which the data is being requested. The join metadata may be, for instance, a bloom filter. The storage system reads the requested data from disk as normal. However, prior to sending the requested data back to the storage system, the storage system analyzes the raw data based on the join metadata, removing a certain amount of data that is guaranteed to be irrelevant to the join operation. The storage system then returns filtered data to the database server. The database system thereby avoids the unnecessary transfer of certain data between the storage system and the database server.

ADAPTIVE SELECTION OF A DISTRIBUTION METHOD DURING EXECUTION OF PARALLEL JOIN OPERATIONS

A method, system, and computer program product for high-performance database systems. The method commences by estimating, during a compile phase, a statically-calculated system cost of performing parallel join operations over two relations, then selecting, during the compile phase, a first distribution method to perform the parallel join operations. Instructions (e.g., table partitioning) are communicated to two or more execution units for execution of the parallel join operations, and during execution, various measurements are taken pertaining to the performance of a first portion of the parallel join operations using the selected first distribution method. Then, responsive to the measurements a second distribution method is selected and corresponding instructions are communicated to the two or more execution units. The parallel join operations continue, now using the second distribution method.

Query task processing based on memory allocation and performance criteria

Described are methods, systems and computer readable media for query task operations based on memory allocation and performance criteria.

REPORTING AND SUMMARIZING METRICS IN SPARSE RELATIONSHIPS ON AN OLTP DATABASE

Storing and/or accessing data in a transactional database, with use of the following technique: (i) selecting a set of base tables in a transactional database; and (ii) creating a factless materialized query table, having maximum sparsity, for the set of base tables. The set of base tables includes at least two base tables. The set of base tables includes a set of keys including at least two distinct primary keys. The factless materialized query table includes one record associated with each record in the set of base tables. Each record in the materialized query table includes a value for every primary key in the set of base tables.

Database system consensus-based access control

A processing system may obtain an operations set associated with database sources of a database system from a client entity, the operations set including a statement, the statement including a query, identify data sets from the operations set, transmit, a request to a first owner to permit access to a first data set, and a request to a second owner to permit access to a second data set, and receive approvals from the first and second owners. The processing system may retrieve a first portion of data stored in the first data set and a second portion of data stored in the second data set in accordance with the approvals, execute the operations set in accordance with the first portion of data and the second portion of data to generate a result set, and provide the client entity access to the result set.

STORAGE RESOURCE PROVISIONING SYSTEMS AND METHODS

A method and apparatus for managing a set of storage resources for a set of queries is described. In an exemplary embodiment, a method provisions processing resources of an execution platform and provisions storage resources of a storage platform. The execution platform uses the storage platform, which is shared with the execution platform, to process the set of queries. The method changes a number of the storage resources provisioned for the storage platform based on a storage capacity utilization by the set of queries of the storage resources. The method changes the number of the storage resources independently to a change of the processing resources in the execution platform. The method processes the set of queries using the changed number of the storage resources provisioned for the storage platform.

JOINING LARGE DATABASE TABLES

Techniques to process a query and perform a join of tables that are distributed across nodes of a network. The join can be performed by analyzing a Where clause. An active flag structure can have flag values that identify table entries satisfying criteria of the Where clause. Keys of surviving entries of a first table can be used to generate a request for a second table to be joined. The request can be for second flags for the second table when the Where clause has criteria for the second table. A response can be used to update the first flags to change a first flag to False. After updating, data can be retrieved for first flags that are True. Requests can use identifiers associated with the first table that identify a location for sending the request, e.g., using RDMA or MPI.

DELAYING EXCEPTIONS IN QUERY EXECUTION

A method includes receiving a query expression indicating an expression evaluation for performance upon output of a filtering operator. An operator execution flow indicating performance of the filtering operator serially after the expression evaluation is generated, and execution of this operator execution flow is facilitated by generating an exception column indicating an exception value generated for each of a set of rows based on performing the expression evaluation. The filtering operator of the operator execution flow can be applied by identifying a subset of the set of rows that meet filtering parameters of the filtering operator. When the exception value of at least one row in the subset indicates an error condition, execution of the operator execution flow is aborted. When the exception value of every row in the subset indicates no error, the subset of the set of rows is outputted and execution of the operator execution flow continues.

Scaling capacity of data warehouses to user-defined levels

A system and method of scaling capacity of data warehouses to user-defined levels. The method includes provisioning a data warehouse including a plurality of processing resources, the plurality of processing resources including at least one processor and at least one storage device. The method includes receiving a request to process database data stored on a storage platform including a plurality of shared storage devices in association with the data warehouse, wherein the request indicates a performance level for processing the request. The method includes determining that a capacity of the plurality of processing resources of the data warehouse would reach a threshold capacity when processing the request according to the performance level. The method includes increasing the capacity of the data warehouse for the data warehouse to process the request according to the performance level.

Stream object type for query processing

Provided herein are systems and methods for a stream object configuration, including configuring and processing streams on views and streams on tables. For example, a method includes parsing, by at least one hardware processor, a received query to determine a plurality of data processing operations associated with a base table. A stream object on the base table is generated. The stream object includes a plurality of query results from executing the plurality of data processing operations and a corresponding plurality of time stamps of times the plurality of query results are obtained. A stream expansion of the stream object is performed. The stream expansion is based on computing changes to the base table from the plurality of data processing operations.

DATABASE ENDPOINT DEVICES PUT IN GROUPS FOR RESPONDING TO QUERIES

Leveraging the competence of a computing cluster and a storage cluster by dynamically adjusting an SQL (structured query language) execution path to optimize system performance with fast response time under a distributed relational database on a data storage platform. Also, endpoint devices are divided into groups for the purpose of responding to queries and/or query blocks. The most suitable group of endpoint devices is chosen to respond to a given query or query block.

Klonen von Katalogobjekten

System zum Klonen von Objekten innerhalb einer Datenbank, wobei das System folgendes umfasst:Mittel zum Identifizieren von mit einer Gruppe von Daten assoziierten Original-Objekten;Mittel zum Erzeugen eine Duplikat-Kopie der mit derselben Gruppe von Daten assoziierten Original-Objekte ohne ein Kopieren der Daten selbst;Mittel zum Zulassen einer Modifikation der mit den Original-Objekten assoziierten Daten unabhängig von den Duplikat-Objekten; undMittel zum Löschen der Duplikat-Objekte auf ein Beenden einer Modifikation der mit den Original-Objekten assoziierten Daten hin.

Systeme zur Bereitstellung von Ressourcen

System, umfassend:eine Ausführungsplattform, die konfiguriert ist, um auf eine Vielzahl von entfernten Speichervorrichtungen zuzugreifen, wobei die Vielzahl von entfernten Speichervorrichtungen kollektiv Datenbankdaten speichert, wobei die Ausführungsplattform eine Vielzahl von virtuellen universellen Datenbanken bzw. Lagern enthält, die jeweils die Vielzahl von entfernten Speichervorrichtungen gemeinsam nutzen, Daten von der Vielzahl von entfernten Speichervorrichtungen wiedergewinnen bzw. abrufen und eine oder mehrere von Client-Vorrichtungen empfangene Anfragen auf den Datenbankdaten ausführen, wobei jedes virtuelle Lager einen Cache, der konfiguriert ist, um von den entfernten Speichervorrichtungen wiedergewonnene bzw. abgerufene Daten zu speichern, und einen Prozessor enthält, der unabhängig von den entfernten Speichervorrichtungen ist; undeinen Ressourcenmanager, der mit der Ausführungsplattform gekoppelt und konfiguriert ist, um:vorhergesagte Datenverarbeitungsaufforderungen auf ...

Transparente Entdeckung eines semistrukturierten Datenschemas

Computerprogramm mit Anweisungen, die dann, wenn sie durch einen oder mehrere Prozessoren ausgeführt werden, veranlassen, dass der eine oder die mehreren Prozessoren Operationen durchführt oder durchführen, die folgendes umfassen:Empfangen semistrukturierter Datenelemente von einer Datenquelle, die über ein Computernetzwerk angeschlossen ist;Durchführen einer statistischen Analyse an Sammlungen der semistrukturierten Datenelemente, wenn sie über einen Computerprozessor zur Datenbank hinzugefügt werden;Identifizieren gemeinsamer Datenelemente von innerhalb der semistrukturierten Daten;Kombinieren gemeinsamer Datenelemente aus der Datenquelle in separate Pseudospalten;Speichern nicht gemeinsamer semistrukturierter Datenelemente in einer serialisierten Überlaufspalte in einem Computerspeicher; undAbleiten von Metadaten entsprechend den Pseudospalten der gemeinsamen Datenelemente aus der statistischen Analyse.

GPU-fähige Datenbanksysteme

Es werden Verfahren zum parallelen Ausführen von Abfragen einer Datenbankoperation in einer eine Grafikverarbeitungseinheit (GPU) verwendenden Datenbank dargestellt, die Folgendes enthalten: Empfangen einer Abfrage durch einen Host-Computer, wobei die Abfrage Datenbankrelationen enthält; Starten eines GPU-Kernel, wobei die GPU-Kernel einen GPU-Speicher enthalten; Hash-Partitionieren der Datenbankrelationen durch den GPU-Kernel; Laden der partitionierten Datenbankrelationen in den GPU-Speicher; Laden von verschlüsselten, den hash-partitionierten Datenbankrelationen entsprechenden Partitionen in den GPU-Speicher; Aufbauen einer Hash-Tabelle für einen kleineren Teil der hash-partitionierten Datenbankrelationen; und Ausführen der Abfrage. In einigen Ausführungsformen enthalten die Verfahren ferner das Zurücksenden eines Ergebnisses der Abfrage. In einigen Ausführungsformen enthalten die Verfahren bei einer langen Abfrage, die mehrere Operationen enthält, ferner das Zergliedern der langen Abfrage ...

GPU enabled database systems

Methods for parallel query execution of a database operation on a database utilizing a graphics processing unit (GPU) are presented including: receiving query by a host, the query including database relations; starting a GPU kernel, where the GPU kernels include a GPU memory; hash partitioning the database relations by the GPU kernel; loading the partitioned database relations into the GPU memory; loading keyed partitions corresponding the hash partitioned database relations into the GPU memory; building a hash table for a smaller of the hash partitioned database relations; and executing the query. In some embodiments, methods further include returning a result of the query. In some embodiments, methods further include when the query is a long query including a number of operators, parsing the long query into a number of sub-queries; for each of the sub-queries, starting one of the GPU kernels such that the sub-queries are processed in parallel.

METHODS AND APPARATUS FOR FUSING DATABASES

Transparent discovery of semi-structured data schema

A system, apparatus, and method for managing data storage and data access for semi-structured data systems.

Resource management systems and methods

Example resource management systems and methods are described. In one implementation, a resource manager is configured to manage data processing tasks associated with multiple data elements. An execution platform is coupled to the resource manager and includes multiple execution nodes configured to store data retrieved from multiple remote storage devices. Each execution node includes a cache and a processor, where the cache and processor are independent of the remote storage devices. A metadata manager is configured to access metadata associated with at least a portion of the multiple data elements.

A BITMAP FILTER, A METHOD OF GENERATING THE SAME, AND A METHOD OF USING A BITMAP FILTER TO PERFORM A JOIN

There is provided a computer-implemented method of generating a bitmap filter. A filter parameter is received, and a first data source associated with the filter parameter is queried to identify at least one entry in the first data source with an identifier corresponding to the filter parameter. A first process is performed where zero or one single bit location of a plurality of bit locations in a bitmap filter is identified that corresponds to an identifier of an entry of the first data source corresponding to the filter parameter. Each identifier has a numerical value and the bit location is identified based on the numerical value of the corresponding identifier. The single bit location is assigned to the identifier, such that there is one-to-one mapping between each identifier corresponding to the filter parameter and an assigned bit location in the bitmap filter. A bit is set at the assigned bit location. The first process is repeated for another entry in the first data source with ...

CHEMICAL SUBSTRUCTURE SEARCH QUERY BUILDING AND EXECUTION

Systems and methods for enabling construction of a complex Boolean chemical substructure query in a structured graphical user interface are provided. The chemical substructures (molecules) may be represented graphically in standard molecular notation, and may be arranged horizontally and vertically on the interface, along with Boolean logical operators. Boolean logical operators of a first type may logically associate molecules arranged in horizontal fashion to form row queries, whereas Boolean logical operators of a different, second type may logically associate the row queries to form a composite query to be applied to a database of molecules. The operators of the first type may comprise disjunctive operators, whereas the operators of the second type may comprise conjunctive operators.

QUERYING JOINED DATA WITHIN A SEARCH ENGINE INDEX

Techniques and systems for indexing and retrieving data and documents stored in a record-based database management system (RDBMS) utilize a search engine interface. Search-engine indices are created from tables in the RDBMS and data from the tables is used to create "documents" for each record. Queries that require data from multiple tables may be parsed into a primary query and a set of one or more secondary queries. Join mappings and documents are created for the necessary tables. Documents matching the query string are retrieved using the search-engine indices and join mappings.

PROCESSING RECORDS FROM A DATABASE

A computer-based method and system of processing records read respectively from first and second related tables of a database includes determining whether data contained in the records read from the first or second tables is unevenly distributed. Records corresponding to evenly distributed data are searched for matches using a first set of instances of the hash-join operator, and records corresponding to data which is unevenly distributed are searched for matches using a second set of instances of a hash-join operator.

AGGREGATING AND UPDATING HETEROGENEOUS DATA OBJECTS

DISTRIBUTED AND OPTIMIZED GARBAGE COLLECTION OF EXPORTED DATA OBJECTS

Described are methods, systems and computer readable media for distributed and optimized garbage collection of remote and exported object handle links to update propagation graph nodes.

ADAPTIVE DISTRIBUTION METHOD FOR HASH OPERATIONS

A method, apparatus, and system for join operations of a plurality of relations that are distributed over a plurality of storage locations over a network of computing components.

HYPER-FOLDING INFORMATION IN A UNIFORM INTERACTION FEED

Embodiments of the present invention provide a computer system, a computer program product, and a method that comprises generating a context data tree for each variable in a plurality of variables based on a received input; determining data folding points for each generated context data tree; conducting a hyper-folding process on the determined data folding points in each context data tree, wherein the hyper-folding process converts each generated context data tree into a single data tree; and automatically loading the single data tree into an application.

Database column refresh via replacement

According to an example, a system may receive a refresh column command for a column-oriented database. The refresh column command may identify a table in the column-oriented database and a denormalized column in the table. The system may determine column definitions for the column, generate a query to populate values from a data source associated with the column into the column based on the column definitions, and may execute the query to replace entire set of existing column values with the data source values.

Query integration across databases and file systems

Query integration across databases and file systems is disclosed. An example method may include streaming data managed by a first database file system for a query. The method may also include streaming data managed by a second database file system for the query. The method may also include joining the streaming data managed by the first database file system with the streaming data managed by the second database file system.

Cross-tenant data leakage isolation

A method and proxy device for cross-tenant data leakage isolation in a multi-tenant database are provided. The method includes monitoring, by a proxy device, traffic flows between a server executing at least one cloud-based application and the multi-tenant database, wherein the proxy device is communicatively connected between the server and the multi-tenant database; capturing, by the proxy device, at least a request to access the multi-tenant database, wherein the request is communicated using a database-specific protocol; analyzing the request to determine if the request is legitimate; upon determining that the request is not legitimate, modifying the request to point to a global-tenant table and to designate a unique tenant identifier, wherein the unique tenant identifier corresponds to a tenant-specific table name designated in the global-tenant table; and sending the modified request to the multi-tenant database using the database-specific protocol.

Spool file for optimizing hash join operations in a relational database system

A method for spooling data for use in joining a small table with a large table in a relational database system. The method analyzes a join condition for combining records from the small and large tables, selects qualified rows from the large table, and writes the qualified rows to a spool file. The spool file includes a first partition containing hash values of all bind terms for the join condition; a second partition including a join column with a best selective bind term; and at least one additional partition including additional join columns used in bind terms. The partitions are grouped together within a container row in the spool file, and multiple container rows are written together within a super-container row in the spool file.

Performing an outer join between a small table and a large table

A small table S is outer joined to a large table L on a join condition on a database system with a plurality B of parallel units (PUs). S and L are partitioned across the PUs. Each row in S has a unique row-id. Each row of S is duplicated on all PUs to form Sdup. On each PU, dangling rows in S that do not have a match in L under the join condition are identified and the row-ids of the dangling rows are saved in Tredis. Tredis is partitioned across the PUs. P is formed from dangling rows of S whose corresponding entries in Tredis appear in all PUs. A result is produced by unioning P and I. I is formed by inner joining non-dangling rows of S with L. The result is saved.

ADAPTIVE DISTRIBUTION FOR HASH OPERATIONS

A method, apparatus, and system for join operations of a plurality of relations that are distributed over a plurality of storage locations over a network of computing components. An example method includes: receiving a relational join query for a join operation associated with a first relation and a second relation; generating at least one build operator and at least one probe operator to perform build operations and probe operations, respectively, of the join operation; and managing a state of one or more communication links between the at least one build operator and the at least one probe operator based on a size of the second relation as determined by the at least one build operator and an estimated size of the first relation.

RESOURCE PROVISIONING SYSTEMS AND METHODS

A method and apparatus managing a set of processors for a set of queries is described. In an exemplary embodiment, a device receives a set of queries for a data warehouse, the set of queries including one or more queries to be processed by the data warehouse. The device further provisions a set of processors from a first plurality of processors, where the set of processors to process the set of queries, and a set of storage resources to store data for the set of queries. In addition, the device monitors a utilization of the set of processors as the set of processors processes the set of queries. The device additionally updates a number of the processors in the set of processors provisioned based on the utilization/Furthermore, the device processes the set of queries using the updated set of processors.

SYSTEM AND METHOD FOR INTEGRATING DATA FOR PRECISION MEDICINE

Methods and systems are provided for a platform and language agnostic method for generating inter-and intra-data type aggregations of heterogeneous disparate data upon which various operations can be performed without altering the structure of the query or resulting distributed data set representation to account for which specific data sources are included in the query.

Resource provisioning systems and methods

A system is described inkling a cache memory and a hardware processor. The hardware processor to determine that at least one of additional data storage capacity and additional processing capacity are needed based on data processing requests of a plurality of computerized query sources to be executed by an execution platform comprising a plurality of nodes, where the data processing requests are directed to database data stored on a plurality of shared storage devices collectively storing the database data. The hardware processor further to increase, in response to the determining, an amount of the data cached by the execution platform by adding one or more nodes to the plurality of nodes.

Equi-joins between split tables

A join operation between split data tables includes providing reduction data from first partitions to each partition among second partitions. The reduction data serves to identify actual values in one of the second partitions that also occur in one of the first partitions. Global IDs are assigned. Translation lists including the global IDs are sent to the first partitions. Each first partition and each second partition create globalized lists which can then be combined to generate respective first and second compiled lists. The join operation can then be conducted on the first and second compiled lists.

ON-THE-FLY ENCODING METHOD FOR EFFICIENT GROUPING AND AGGREGATION

Embodiments include a method, system, and computer program product for encoding data while it is being processed as part of a query is provided. The method includes receiving a query request and determining a set of values associated with data to be encoded for completing the query request. The method also includes encoding those values such that any subsequent processing operations can be performed on the encoded values to complete the requested query. After performing the subsequent processing operations to complete the requested query, each value is decoded back to its original value.

GENERATING REAL-TIME AGGREGATES AT SCALE FOR INCLUSION IN ONE OR MORE MODIFIED FIELDS IN A PRODUCED SUBSET OF DATA

A data processing system for producing a subset of data from a plurality of data sources, including: memory storing a plurality of data sources to be represented in an editor interface; a data structure modification module that selects a plurality of data sources to be represented in an editor interface and generates a subset of data included in the plurality of data sources; memory that stores the selected data structures included in the subset, with at least one of the stored data structures including the one or more modified attributes of the one or more respective fields; rendering module that displays, in the editor interface, representations of the stored data structures; and a segmentation modules that segments a plurality of received data records.

Systems and methods for managing shared content based on sharing profiles

Content items stored in an online content management service can be organized and shared. Content items can be associated with sharing profiles that include various sharing-specific metadata, such as details of how an item is shared or with whom it is shared. In some embodiments, the metadata stored in the sharing profiles can be used to organize shared content into shared folders automatically and/or to sort a list of content items.

Query generation from a natural language input

A query generation system receives, from a first device, a first input and a first project identifier and receives, from a second device, a second input and a second project identifier. The first and second inputs are the same and are in a natural language format that is not compatible with a downstream database management system. The system generates, based on the first input, a first database query. The system generates, based on the second input, a second database query. The first and second database queries are compatible with the downstream database management system. The system receives a first response to the first database query and a second response to the second database query from the downstream database management system. The system transmits the first response to the first device and the second response to the second device.

Distributed pseudo-random subset generation

Distributed pseudo-random subset generation includes obtaining a data-query indicating a first table having a first column including unique values, a second table having a second column including unique values, a join clause joining the first table and the second table on the first column and the second column, and a limit value, pseudo-random filtering the first table to obtain left intermediate data and left filtering criteria, pseudo-random filtering the second table to obtain right intermediate data and right filtering criteria, obtaining intermediate results data by full outer joining the left intermediate data and the right intermediate data, obtaining results data by filtering the intermediate results data using most-restrictive filtering criteria among the left filtering criteria and the right filtering criteria, and outputting the results data, wherein outputting the results data includes limiting the cardinality of rows of the results data to be at most the limit value.

Single input graphical user interface control element and method

Described are methods, systems and computer readable media for GUI control elements and associated processing methods.

Nachverfolgen von zwischenzeitlichen Änderungen bei Datenbankdaten

Computerprogramm, das Anweisungen enthält, die dann, wenn sie durch einen oder mehrere Prozessoren ausgeführt werden, veranlassen, dass der eine oder die mehreren Prozessoren folgendes veranlasst oder veranlassen:Unterhalten von gespeicherten Daten in einer ersten Mikropartition einer Datenstruktur, wobei die gespeicherten Daten mit einem ersten Zeitstempel assoziiert sind;in Reaktion auf eine Transaktion Erzeugen von Transaktionsdaten, die mit der Transaktion assoziiert sind, und Durchführen von einer oder mehreren Modifikationen an den gespeicherten Daten in der ersten Mikropartition, um eine zweite Mikropartition basierend auf wenigstens der einen oder den mehreren Modifikationen an den gespeicherten Daten zu erzeugen, wobei die zweite Mikropartition bei einem zweiten Zeitstempel erzeugt wird;Erzeugen von Modifikationsdaten, wobei die Modifikationsdaten die an den gespeicherten Daten in der ersten und der zweiten Mikropartition zwischen dem ersten und dem zweiten Zeitstempel durchgeführten ...

JOIN OPERATION PROCESSING SYSTEM

DML statements for densifying data in a relational database system

Generating data visualizations according to an object model of selected data sources

The method receives a visual specification, which specifies a data source, visual variables, and data fields from the data source. Each visual variable is associated with data fields and each data field is either a dimension or a measure. From an object model of the data source, the method identifies a minimal subtree that includes all of the dimension data fields and constructs a query from the minimal subtree. The method executes the query against the data source to retrieve a set of tuples, each tuple comprising a unique ordered combination of data values for the dimension data fields. For each tuple, the method forms an extended tuple by appending aggregated data values corresponding to each measure data field. The method then builds and displays a data visualization according to the data fields in the extended tuples and according to the visual variables to which the data fields are associated.

Secure equijoin system, secure equijoin device, secure equijoin method, and program

The present invention provides a secret equi-join technique for creating one table from two tables while minimizing the amount of communication. A secret equi-join device includes: a first substitution ...

BUDGET TRACKING IN A DIFFERENTIALLY PRIVATE DATABASE SYSTEM

Techniques are described for budget tracking in a differentially private security system. A request to perform a query of a private database system is received by a privacy device from a client device. The request is associated with a level of differential privacy. A privacy budget corresponding to the received request is accessed by the privacy device. The privacy budget includes a cumulative privacy spend and a maximum privacy spend, the cumulative privacy spend representative of previous queries of the private database system. A privacy spend associated with the received request is determined by the privacy device based at least in part on the level of differential privacy associated with the received request. If a sum of the determined privacy spend and the cumulative privacy spend is less than the maximum privacy spend, the query is performed. Otherwise a security action is performed based on a security policy.

SYSTEM AND METHOD FOR VISUAL BAYESIAN DATA FUSION

System and method for visual Bayesian data fusion are disclosed. In an example, a plurality of datasets associated with a topic are obtained from a data lake. Each of the plurality of datasets include information corresponding to various attributes of the topic. Further, the plurality of datasets are joined to obtain a joined dataset. Furthermore, distribution associated with a target attribute is predicted using Bayesian modeling by selecting a plurality of attributes (k) based on mutual information with the target attribute in the joined dataset, learning a minimum spanning tree based Bayesian structure using the selected attributes and the target attribute, learning conditional probabilistic tables at each node of the minimum spanning tree based Bayesian structure; and predicting the distribution associated with the target attribute by querying the conditional probabilistic tables, thereby facilitating visual Bayesian data fusion.

AGGREGATING AND UPDATING HETEROGENEOUS DATA OBJECTS

JOINT MODEL AGNOSTIC AGGREGATION COMPUTATION IN DATABASE QUERY OPERATIONS

Merging partial query results into a single result

A method and system for executing an information retrieval query in a multiserver computing environment is disclosed. The method and system employ a technique in which the query is distributed among each of a plurality of partial index servers in the multiserver environment, and a subset of results is calculated for each of the plurality of partial index servers. Then, the subset of results are merged in one logical index server to generate a merged result.

Push model for intermediate query results

A system and method for managing data storage and data access with querying data in a distributed system without buffering the results on intermediate operations in disk storage.

HYBRID EXECUTION OF CONTINUOUS AND SCHEDULED QUERIES

Techniques for implementing the hybrid execution of continuous and scheduled queries are provided. In some examples, a query engine may be initialized with relational data from at least a first source. For example, the first source may include a database or other system that can provide historical data. Additionally, the query engine may be enabled to provide query results based at least in part on the relational data from at least the first source and streaming data from at least a second source. In some examples, the second source may be a data stream.

OPTIMIZING WINDOW JOINS OVER DATA STREAMS

Optimizing window joins over data streams can include receiving an input topology and calculating costs of computing a join of data streams based on a number of model topologies, and determining an optimal topology based on the calculated costs and the input topology, wherein the input topology, the model topologies, and the optimal topology each include a number of interconnected operators.

Incrementally building hash collision tables

Disclosed herein are system, method, and computer program product embodiments for incrementally building hash collision tables. In some embodiments, hashes and hash collision tables may be used to improve efficiency of relational operations, such as those used in relational databases. An embodiment operates by determining hash collisions between data entries, then storing newly determined hash values corresponding to collisions in a hash collision table. The storing can be done incrementally, without needing to rebuild hash tables or hash collision tables for all data entries.

Systems, methods, and devices for managing data skew in a join operation

Systems, methods, and devices, for managing data skew during a join operation are disclosed. A method includes computing a hash value for a join operation and detecting data skew on a probe side of the join operation at a runtime of the join operation using a lightweight sketch data structure. The method includes identifying a frequent probe-side join key on the probe side of the join operation during a probe phase of the join operation. The method includes identifying a frequent build-side row having a build-side join key corresponding with the frequent probe-side join key. The method includes asynchronously distributing the frequent build-side row to one or more remote servers.

Merging apparatus and merging method

A merging apparatus includes a first database in which a plurality of attributes including a key attribute that is set in advance as a key for merging is set and a second database in which one or a plurality of attributes excluding the key attribute is set. Furthermore, the merging apparatus checks an attribute that is set in the first database against an attribute that is set in the second database and extracts a common attribute that is set in common in both the first database and the second database. Then, the merging apparatus merges the first database with the second database by determining whether attribute values classified under the extracted common attribute match.

IMPLEMENTING LINEAR ALGEBRA FUNCTIONS VIA DECENTRALIZED EXECUTION OF QUERY OPERATOR FLOWS

A method for execution by a query processing system includes determining a query request that indicates a plurality of operators, where the plurality of operators includes at least one relational algebra operator and further includes at least one non-relational operator. A query operator execution flow is generated from the query request that indicates a serialized ordering of the plurality of operators. A query resultant of the query is generated by facilitating execution of the query via a set of nodes of a database system that each perform a plurality of operator executions in accordance with the query operator execution flow, where a subset of the set of nodes each execute at least one operator execution corresponding to the at least one non-relational operator in accordance with the execution of the query.

EVALUATING QUERY TRANSLATIONS FOR CROSS-LANGUAGE QUERY SUGGESTION

Computer-implemented methods, systems, computer program products for generating cross-language query suggestions are described. For each query suggestion written in a first natural language, candidate segmentations are generated from the query suggestion, and candidate translations are generated from each candidate segmentation. The candidate translations are evaluated based on a measure of segmentation quality associated with the respective candidate segmentation from which each candidate translation is derived, and a frequency of occurrence of the candidate translation in a target language query log. The measure of segmentation quality associated with each candidate segmentation is further based on a frequency of occurrence of the candidate segmentation in a source language query log. A candidate translation is provided as a cross-language query suggestion for the primary language query suggestion based on the result of the evaluation.

For all entries processing

A system includes determination of a value identifier for each value of a first table, based on a data dictionary of a second table, generation of a hash function, the hash function mapping, for each row of the first table, one or more value identifiers of one or more columns of the row to a pointer to the row, application of the hash function to one or more value identifiers of a row of a second table, determination of whether the output of the hash function is a valid hash, and, if it is determined that the output of the hash function is a valid hash, determination that the row of the second table is associated with the first table.

Flexibly performing allocations in databases

Allocation within a database environment is performed by processing a sequence of steps, each of which is described by an allocation processing block. The allocation processing block receives three inputs (input tables or input view) and computes a result. Related apparatus, systems, techniques and articles are also described.

Transforming a scalar subquery

A method, computer system, and a computer program product for transforming a scalar sub-query to join with non-provable singleton output, responsive to receiving a parsed representation of a statement, containing a scalar operation, which is correlated to predetermined types of objects, by analyzing the parsed representation to determine eligibility for transformation of any scalar operations. Responsive to a determination of eligibility, a query engine injects additional processing into the parsed representation to defer checking of a 1 row condition required by statement semantics. The query engine further injects processing to utilize a result of the additional processing to validate a single row is produced for each GROUPING/PARTITION key. The parsed representation is modified to remove the 1 row condition. The query engine decorrelates the parsed representation using a predetermined method absent a requirement for a unique key to be present in a predicate of the scalar operation.

Decomposition of queries

Techniques are disclosed to decompose a query from a client application. Embodiments include receiving a query based on a request provided by a user via a user interface. Embodiments include identifying a set of data elements requested by the query and identifying a predicate expression of a plurality of expressions in the query and a dependent expression of the plurality of expressions in the query. Embodiments include generating a first micro-service query for at least a first data element corresponding to the predicate expression and a second micro-service query for at least a second data element corresponding to the dependent expression. Embodiments include determining an order for issuing the first micro-service query and the second micro-service query based on the predicate expression and the dependent expression.

Data overlap count adjustment in a multiple tenant database system

Systems, methods, and devices for generating a secure join of database data are disclosed. A method creates a secure view of datapoints of a consumer account and processes, using a secure user defined function (UDF), the datapoints of the consumer account and datapoints of a provider account to generate a secure join key. The datapoints of the consumer account are provided to the secure UDF using the secure view. The method further performs, by a processor, an analysis of the datapoints of the consumer account and the datapoints of the provider account of the secure join key. The analysis returns a count value of overlapping datapoints between the consumer account and the provider account. The method further adjusts the count value of overlapping datapoints based on a number of distinct rows associated with the provider account, and provides the adjusted count value of overlapping datapoints to the consumer account.

Query processing distribution

Example caching systems and methods are described. In one implementation, a method identifies multiple files used to process a query and distributes each of the multiple files to a particular execution node to execute the query. Each execution node determines whether the distributed file is stored in the execution node's cache. If the execution node determines that the file is stored in the cache, it processes the query using the cached file. If the file is not stored in the cache, the execution node retrieves the file from a remote storage device, stores the file in the execution node's cache, and processes the query using the file.

Digital processing systems and methods for communications triggering table entries in collaborative work systems

Systems, methods, and computer-readable media for triggering table entries characterizing workflow-related communications occurring between workflow participants are disclosed. The systems and methods may involve presenting a table via a display, the table containing rows and columns defining cells, the rows and cells being configured to manage respective roles of the workflow participants; presenting on the display at least one active link for enabling workflow participants to join in a video or an audio communication; logging in memory, characteristics of the communication including identities of the workflow participants who joined in the communication; and generating an object associated with the table, the object containing the characteristics of the communication logged in memory.

System and method for querying a data repository

A method is disclosed, as well as systems, performed by one or more processors, for interacting with data in a data repository. The method comprises receiving, in a data catalogue environment, a search request relating to one or more items in the data repository and determining an object type associated with the one or more items. Other operations comprise loading an object template in dependence on the determined object type, populating the template with data from the data repository in dependence on the search request to create an object view, and displaying the object view within the data catalogue environment. The data repository comprises a plurality of joined datasets, and wherein the object view comprises one or more links to items in a joined dataset.

METHOD FOR GENERATING VIEWS BASED ON A SEMANTIC MODEL, THAT ALLOWS FOR AUTONOMOUS PERFORMANCE IMPROVEMENTS AND COMPLEX CALCULATIONS

Herein are techniques for automatically leveraging metadata of an analytic view to accelerate a relational query. In an embodiment, a computer stores model metadata that defines an analytic view that contains a join operation that is based on a dimension column of a dimension table and a join column of a fact table. The analytic view also contains a measure that is based on an aggregation operation and a measure column of the fact table. Also stored is denormalization metadata that defines a transparency view that is based on the analytic view. In operation, a query that references the transparency view is received. The query does not reference the analytic view. The query that references the transparency view is executed based on: a) the denormalization metadata that defines the transparency view, b) the model metadata that defines the analytic view, and c) the measure that is based on the aggregation operation and the measure column of the fact table.

Systems And Methods For Securing Data Based On Discovered Relationships

Techniques for automatically discovering and protecting sensitive data are disclosed. In some embodiments, a set of data objects is searched for data matching a first set of one or more regular expressions and for metadata matching a second set of one or more regular expressions. A confidence score is then generated for a particular data objects in the set of data objects as a function of regular expressions in the first set of one or more regular expressions that match data stored in the particular data object and regular expression in the second set of one or more regular expressions that match metadata associated with the particular data object. One or more operations may be performed to protect sensitive data stored in the particular data object based, at least in part, on the confidence score.

System and methods for joining datasets

An intermediate mapping entity storing association between the keys of a first type used for a first dataset and a second type used for the second dataset is selected based on the intersection weight (the proportion of overlapping data entries) between the first and second data sets. This enables joining of the datasets, for example, medical or financial data where there is no direct correlation between the data. The mapping entities define multiple paths from the first to the second data. A best single path may be selected based on optimising a heuristic of the set of path parameters.

A database query and visualisation system

A database query and visualisation system provides an interface with which a user can specify one or more queries. The system generates a response to each query which is displayed as a table, graph or other visual representation. The system is configured to process each query as a layer, where a layer is a node in a directed graph, and a set of nodes forms a notebook. The notebook is therefore made up of one or more layers, and each layer generally corresponds to a step in larger, more complex query. The notebook can be presented vertically down a scrollable page or as a dashboard. Changes made in one layer can cascade to other layers which depend on the results of that layer. A query in a layer, which may be a natural language query, is transformed by the system into an exact, unambiguous, query in a database query language, such as SQL.

System, method and computer readable medium for binding authored content to the events used to generate the content

A web page that includes content form fields may be modified to include an event observer module and an authored content module. Events generated during the authoring of content by a user are recorded by the event observer module and sent to an event server with an InteractionID. The authored content module inserts hidden fields into the form fields that are updated with the InteractionID when content is submitted to the web server. The web server provides the InteractionID in a bind request to the event server. The event server binds the content to the events used to create the content in response to the request.

Complex event processing apparatus and complex event processing method

When a detecting complex event condition expression is changed, a rule comparing unit compares the complex event condition expressions before and after the change. The changed portion identifying unit identifies the changed portion based on the comparison result, and the parallel operating unit operates the complex event condition expressions before and after the change in parallel for the detecting complex event condition expression including the identified changed portion. In this manner, the complex event processing apparatus disclosed therein can dynamically change a detecting complex event condition expression used in the complex event processing.

Reading rows from memory prior to reading rows from secondary storage

In an embodiment, a first plurality of rows of a first table are read from memory in an order of page addresses of pages in the memory that comprise the first plurality of rows. First selected rows from the first plurality of rows that meet a query predicate are stored into a result set. After the reading the first plurality of rows in the order of the page addresses, a second plurality of rows of the first table are read from secondary storage, in an order of row identifiers of the second plurality of the rows in the first table. Second selected rows from the second plurality of rows that meet the query predicate are stored into the result set.

Numeric, decimal and date field compression

A method, apparatus, and article of manufacture for accessing data in a computer system. Compression and decompression functions are associated with a column of the table, in order to perform compression of decimal, numeric or date data stored in the column when the data is inserted or updated in the table, and in order to perform decompression of the data stored in the column when the data is retrieved from the table. The compression function compresses and stores the data in a fixed-length compressed field in the column without a length value, and the fixed-length compressed field has a size that is determined by a range of values for the data stored in the fixed-length compressed field. The decompression function retrieves and decompresses the data from the fixed-length compressed field.

Data staging for results of analytics

Systems and methods of data staging for results of analytics are disclosed. An example of a method includes maintaining current results from the analytics in a first data structure, the first data structure having a label identifying first data structure as a target for queries. The method also includes maintaining prior results from the analytics in at least one other data structure. The method also includes changing the label of the first data structure after a predetermined time. The method also includes assigning the label to one of the other data structures, wherein the label identifies the one of the other data structures as the target for queries.

Active learning of record matching packages

An active learning record matching system and method for producing a record matching package that is used to identify pairs of duplicate records. Embodiments of the system and method allow a precision threshold to be specified and then generate a learned record matching package having precision greater than this threshold and a recall close to the best possible recall. Embodiments of the system and method use a blocking technique to restrict the space of record matching packages considered and scale to large inputs. The learning method considers several record matching packages, estimates the precision and recall of the packages, and identifies the package with maximum recall having precision greater than equal to the given precision threshold. A human domain expert labels a sample of record pairs in the output of the package as matches or non-matches and this labeling is used to estimate the precision of the package.

Database acceleration using gpu and multicore cpu systems and methods

A computer-implemented method for GPU acceleration of a database system, the method includes a) executing a parallelized query against a database using a database server, the parallelized query including an operation using a particular stored procedure available to the database server that includes a GPU/Many-Core Kernel executable; and b) executing the particular stored procedure on one or more GPU/Many-Core devices.

Join order restrictions

A query that is submitted to a database is analyzed so as to determine a join order restriction. The join order restriction is associated with a join between two tables of a plurality of tables of the database that participate in the query. The join order restriction constrains its associated join to be executed prior to another join of the query. At least one join plan is generated, the join plan being constrained by the join order restriction. Different join plans include different join orders. A join plan is selected from among the join plans for execution of the query.

Accelerated Join Process in Relational Database Management System

Techniques are provided for an accelerated join process in a relational database management system. The disclosed join method partitions a plurality of input records using a hash-based technique to form a plurality of partitioned blocks. The partitioned blocks are sorted to form sorted partitioned blocks. The sorted partitioned blocks are then compressed to form a plurality of compressed blocks of records. The compressed blocks of records are stored for each partition in a storage system. The compressed blocks of records associated with a pair of partitions can then be loaded into a main memory. The loaded compressed blocks of records are then decompressed and the decompressed blocks of records are merged into associated merged partitions. Finally, two of the merged partitions are joined by comparing records from each merged partition. In a multi-threaded implementation, multiple pairs of merged partitions can be joined in parallel.

Value-based positioning for outer join queries

Enterprise systems, methods and computer program products are disclosed for providing value-based positioning for outer join database queries in an enterprise resource planning system operating on a specially programmed computer server. The computer implemented method comprises; receiving a query at a server and processing the query for inner join and then evaluating a returned value for Order by direction and whether the current row returns NULL. Depending on forward paging and reverse paging for outer join, the process involves changing an inequality operator depending on the evaluated Order by direction and whether the current row is NULL. The process for inner join and outer join queries is implemented on an application object server including a kernel instance, a query framework and query generation model and a data access layer engine to a database of the enterprise resource planning system.

Sorting a dataset of incrementally received data

A method of sorting a dataset includes incrementally receiving data from the dataset, and incrementally storing the received data as individual input data subsets as the data is received, thereby sequentially generating a plurality of filled data subsets of unsorted data. The method includes individually sorting each filled data subset of unsorted data concurrently with receiving data for a next one of the individual input data subsets, thereby sequentially generating a plurality of sorted input data subsets, and performing a merge sort on the plurality of sorted input data subsets, thereby incrementally generating a sorted version of the dataset.

Generating discrete event simulation data

A method of generating discrete event simulation data using a query is provided herein. The method includes retrieving a trajectory specifying a service access event for a customer accessing a service in a simulation. The method also includes processing, in a relational database query, one or more pending events corresponding to one or more customers accessing the service before the customer. Additionally, the method includes processing, in the relational database query, the service access event. Processing the service access event occurs after processing the one or more pending events. The method also includes providing a result for the relational database query. The result comprises the trajectory, and the one or more trajectories.

Dynamic reduction of stream backpressure

Techniques are described for eliminating backpressure in a distributed system by changing the rate data flows through a processing element. Backpressure occurs when data throughput in a processing element begins to decrease, for example, if new processing elements are added to the operating chart or if the distributed system is required to process more data. Indicators of backpressure (current or future) may be monitored. Once current backpressure or potential backpressure is identified, the operator graph or data rates may be altered to alleviate the backpressure. For example, a processing element may reduce the data rates it sends to processing elements that are downstream in the operator graph, or processing elements and/or data paths may be eliminated. In one embodiment, processing elements and associate data paths may be prioritized so that more important execution paths are maintained. In another embodiment, if a request to add one or more processing elements may cause future backpressure, the request may be refused.

Systems and/or methods for forecasting future behavior of event streams in complex event processing (cep) environments

Certain example embodiments described herein relate to forecasting the future behavior of event streams in Complex Event Processing (CEP) environments. For each received event in an event stream, a reference window indicative of a predefined temporal range during which the forecast is to be computed is updated so that the reference window ends with the received event, with the reference window moving with the event stream. Within this processing loop, when a forecasting update policy indicates that the forecast is to be updated based on the received event: a forecasting window indicative of a temporal range in which events are to be forecasted is updated; and while the time period of the forecasting window is not exceeded, (a) a next forecasted event is generated via at least one processor and (b) the next forecasted event is inserted into the forecast window; and the forecast window is published.

Real time searching and reporting

A system arranged to search machine data to generate reports in real time. A search query is provided that includes a plurality of search commands. The search query is parsed to form a main search query and a remote search query. Machine data is collected from remote data sources and evaluated against one of the main and remote search queries to generate a set of search results. The main search query is then evaluated against at least a partial set of the search result to generate at least one report regarding the collected machine data. Initially a search window is pre-populated with historical machine data related to the search query. Over time the historical machine data is replaced with the collected machine data.

Dynamic reduction of stream backpressure

Techniques are described for eliminating backpressure in a distributed system by changing the rate data flows through a processing element. Backpressure occurs when data throughput in a processing element begins to decrease, for example, if new processing elements are added to the operating chart or if the distributed system is required to process more data. Indicators of backpressure (current or future) may be monitored. Once current backpressure or potential backpressure is identified, the operator graph or data rates may be altered to alleviate the backpressure. For example, a processing element may reduce the data rates it sends to processing elements that are downstream in the operator graph, or processing elements and/or data paths may be eliminated. In one embodiment, processing elements and associate data paths may be prioritized so that more important execution paths are maintained.

Stream application performance monitoring metrics

Techniques are disclosed for calculating performance metrics associated with a data stream. A processing element configured to process data tuples flowing through a stream-based computing system receives data tuples via the data stream. Each of the one or more tuples is processed at the processing element, and each of the one or more tuples are associated with metadata that includes information related to the processing of the tuple by the processing element. Performance metrics are then calculated for the data stream based on the metadata associated with the one or more tuples.

User interface displaying filtered information

A trigger event is set, based on information in an information flow. One or more actions are set to occur in response to occurrence of the trigger event. The information flow is received. The set one or more actions are performed upon occurrence of the trigger event, and a user interface is displayed based on the information flow. The actions include, but are not limited to, filtering display of information from the information flow in response to occurrence of the trigger event.

Content management system that retrieves data from an external data source and creates one or more objects in the repository

A content management system (CMS) provides a way to include data from an external source as an object in the repository managed by the CMS. The CMS includes an external data mechanism that processes a document with a query to a data source external to the repository. The external data mechanism executes the query, and creates one or more objects in the repository according to the type of data returned. Each object containing data returned from the query is linked to that query via a formal relationship in the CMS. When an object that has a formal relationship with a query is checked out of the repository, the query corresponding to the object may be executed, and the data returned from the query may be stored in the object. Alternatively, the data may be used directly if a corresponding lifecycle policy indicates the corresponding query should not be executed.

Content management system that retrieves data from an external data source and creates one or more objects in the repository

A content management system (CMS) provides a way to include data from an external source as an object in the repository managed by the CMS. The CMS includes an external data mechanism that processes a document with a query to a data source external to the repository. The external data mechanism executes the query, and creates one or more objects in the repository according to the type of data returned. Each object containing data returned from the query is linked to that query via a formal relationship in the CMS. When an object that has a formal relationship with a query is checked out of the repository, the query corresponding to the object may be executed, and the data returned from the query may be stored in the object. Alternatively, the data may be used directly if a corresponding lifecycle policy indicates the corresponding query should not be executed.

Intelligent event query publish and subscribe system

Indexing and routing to event data is described. Event data is assigned an identifier that identifies the data type and the contents of event data within an enterprise system. The event data may be real-time event data. With the identifier, a source of the event data is determined, and the source can be queried for the event data in real-time. The identifier is indexed along with other event data identifiers. Based on the location of the event data, the system sends out a query toward the data source to obtain the information, but also to route the query to the data source, rather than attempting to pull data towards the query source and process it at the query source.

Techniques for updating join indexes

Techniques for updating join indexes are provided. A determination is made to update date criteria in a join index query statement. The join index is parsed for current date and current time criteria. The join index is revised based on the location of the current date and current time criteria as they appear in the original join index. The revisions include new criteria that minimize the effort in maintaining and using the join index.

Systems and methods for data brick creation and use

Data Brick is a state of the art solution to access the data through high speed data access mechanism that is exceptionally fast and is associated with low CPU consumption cost. The Data Brick stores relatively static reference data in the form of a programming language Load Module. The Load module contains the data can be launched into the application program memory and this data can be accessed very efficiently in a programming language program. Static application data that is accessed by multiple applications in a mainframe Batch environment is a prime candidate for the Data Brick.

System and Method for Querying a Data Stream

There is provided a method ( 200 ) for querying a data stream. The method includes receiving a query plan based on a query specifying the data stream and a window. The method ( 200 ) further includes receiving one or more stream elements from the data stream during the window. Additionally, the method ( 200 ) includes applying the query to the one or more stream elements by passing the one or more stream elements from a scan operator at a leaf of the query plan to an upper layer of the query plan on a tuple-by-tuple basis. The method ( 200 ) also includes committing a result of the query based on the one or more stream elements.

Continuous Querying of a Data Stream

In continuous querying of a data stream, a query including query cycles can be initialized ( 310 ) on a query engine to analyze the data stream for desired information. The data stream can be processed ( 320 ) as segments, where a size of the segments is based on a user-defined parameter. The query cycles can be synchronized ( 330 ) with the segments of the data stream. A first segment can be analyzed ( 340 ) by performing the query on the first segment to obtain a first result. A query state of the query can be persisted ( 350 ) and the query operation can be rewound to begin a new query cycle. A second segment can be analyzed ( 360 ) in the new query cycle by performing the query on the second segment based on the first result.

System and program for joining source table rows with target table rows

A data processing system, and an article of manufacturing, join rows associated with a source table column with rows associated with a target table column. A source node and a target node contain the source and target tables, respectively. A reduced representation of selected rows associated with the source table column is generated, as is a representation of the target table column. A filtering module filters the generated reduced representation of selected rows associated with the source table column through the generated representation of the target table column, the filtered generated reduced representation of selected rows identifying source table rows that do not have to be joined with the target table. The rows associated with the source table column minus the filtered generated reduced representation of selected rows are joined to the rows associated with the target table column.

Systems and methods for performing a nested join operation

Systems and methods are provided for performing a nested join operation. Partitioning key values are computed for an outer data source using a partitioning key function used to partition an inner table. A join process is established for each of a plurality of partitions of the inner table (at 204 ), with a given partition of the inner table representing a plurality of partitioning key values. Each row from the outer data source is routed to a join process according to its associated partitioning key value (at 206 ). The inner table is probed to return a row from the inner table having the partitioning key value associated with the row from the outer data source (at 208 ). The row from the outer data source and the row returned from the inner table are joined to form a row in a combined table (at 210 ).

Method and apparatus for aggregating server based and lan based media content and information for enabling an efficient search

A method and apparatus for aggregating server based and LAN based media content and information for enabling an efficient search include discovering local and external content directory service instances, storing at least one of content of the discovered content directory service instances and metadata identifying content available via the discovered content directory service instances in a common database and providing a user interface such that a user is able to search for content across the discovered content directory service instances. In one embodiment a common database comprises a de-serialized database which provides use of indexes for enabling searches.

Population clustering through density based merging

A method and/or system for analyzing data using population clustering through density based merging.

Indicating Bit Stream Subsets

A method of indicating bit stream subsets in a video bit stream ( 210 ) is provided. The method comprises receiving the bit stream, dividing the bit stream into video packets ( 211 - 216 ), wherein each packet comprises either one of video data or supplemental information, and marking each packet with a single subset identifier (stream_id). Each subset identifier is associated with a corresponding bit stream subset ( 221 - 223 ). Further, a method of extracting video packets from a video bit stream is provided. The method comprises providing relevant subset identifiers, receiving video packets from the bit stream, and, for each received packet, inspecting the subset identifier of the packet. The packet is extracted if the subset identifier matches one of the relevant subset identifiers. This allows condensing properties of a bit stream subset into a single identifier, thereby simplifying the processing of video packets in the network and on the client side. Further, devices corresponding to the aforementioned methods are provided.

Data stream quality management for analytic environments

According to one aspect of the present disclosure, a system and technique for data quality management is disclosed. The system includes a processor and an ingress quality specification (IQS) module executable by the processor in a runtime environment with a data stream analytic module. The IQS module is configured to: receive the data stream; analyze a subset of data of the data stream to determine if the subset of data meets a quality expectation of the analytic module; annotate the subset of data to indicate a quality status based on whether the subset of data meets the quality expectation of the analytic module; and output the data stream to the analytic module.

Metadata-based Test Data Generation

Example embodiments disclosed herein relate to a method for generating test data for form validation, where the method includes maintaining metadata for a database, mapping a control field of the form to a column of the database, and generating test data based on the mapping and the metadata.

Automatic Denormalization For Analytic Query Processing In Large-Scale Clusters

A system and method for improving performance of join operations is disclosed. The method in one embodiment includes building a graph of vertices and edges, which represent the tables and join conditions for a particular database schema. The method then traverses the graph to find a set of candidate join operations. The method further includes selecting from the set of subset of join operations that makes the best use of a given amount of memory, which means finding a collection of denormalizations which when residing in memory provide the most benefit and the least cost for the given amount of memory. The method then deploys the denormalizations so that current queries in a query processing system can use them instead of actually performing the joins.

Facilitating dynamically controlled fetching of data at client computing devices in an on-demand services environment

In accordance with embodiments, there are provided mechanisms and methods for facilitating dynamically controlled fetching of data at client computing devices in an on-demand services environment in a multi-tenant environment according to one embodiment. In one embodiment and by way of example, a method includes receiving, at a server computing device, a request from a client computing device. The request includes a query for receiving data from a database coupled to the server computing device in an on-demand database system. The method may further include matching the request against a predetermine threshold value, and where the threshold value includes a first predetermined threshold range based on a first column range or a first row range and a second predetermined threshold range based on a second column range or a second row range. The method may further include generating multiple responses to the request based on one or more formulae and the predetermined threshold value if the request fails to satisfy the predetermined threshold value.

Anomaly, association and clustering detection

Techniques are provided for anomaly, association and clustering detection. At least one code table is built for each attribute in a set of data. A first code table corresponding to a first attribute and a second code table corresponding to a second attribute are selected. The first code table and the second code table are merged into a merged code table, and a determination is made to accept or reject the merged code table. An anomaly is detected when a total compression cost for a data point is greater than a threshold compression cost inferred from one or more code tables. An association in a data table is detected by merging attribute groups, splitting data groups, and assigning data points to data groups. A cluster is inferred from a matrix of data and code words for each of the one or more code tables.

Detection and handling of aggregated online content using characterizing signatures of content items

A computer-implemented method is disclosed herein. The method obtains a first content item from an online source, and selects only a portion of the words in the first content item, in accordance with a defined selection algorithm. The method continues by generating a characterizing signature of the first content item, wherein the characterizing signature is derived from the selected words. The method continues by finding a previously-saved instance of the characterizing signature, retrieving data associated with a second content item (which is linked to the saved characterizing signature), and comparing the data associated with the second content item against corresponding data associated with the first content item. The first content item or the second content item is identified as an original content item, based on the comparing. The other content item is flagged as an aggregated content item.

Handling out-of-sequence data in a streaming environment

Computer-implemented method, system, and computer program product for processing data in an out-of-order manner in a streams computing environment. A windowing condition is defined such that incoming data tuples are processed within a specified time or count of each other. Additionally, the windowing condition may be based on a specified attribute of the data tuples. If the tuples are not processed within the constraints specified by the windowing condition, the unprocessed tuples may be discarded, i.e., not processed, to optimize operator performance.

State initialization algorithm for continuous queries over archived relations

In some examples, a method may include receiving a continuous query, the continuous query being identified based at least in part on an archived relation associated with an application. The method may also include generating a physical query plan for the continuous query. In some examples, the method may also include identifying a query operator in the physical query plan and/or constructing an archiver query for the identified query operator. In some aspects, the method may also include executing the archiver query to obtain a result set of data records related to the application and/or initializing a state of the query operator based at least in part on the result set of data records. Further, the method may also include generating a snapshot output of one or more data values related to the application based at least in part on the state of the identified query operator.

Join operations for continuous queries over archived views

A continuous query may be received, the continuous query being identified based at least in part on an archived view. The archived view may be created and/or identified based at least in part on a join query related to two or more archived relations associated with an application, at least one of the two or more archived relations being identified as a dimension relation. A query plan for the continuous query may be generated. A join operator in the query plan may be identified based at least in part on the dimension relation. A state of an operator corresponding to the dimension relation may be initialized. It may be identified if the state of the operator identifies an event that detects a change to the dimension relation. The continuous query may be re-started based at least in part on the event that detects the change to the dimension relation.

Fast path evaluation of boolean predicates

Techniques for managing fast path evaluation of Boolean predicates are provided. In some examples, the fast path evaluation may be based at least in part on received queries and/or query statements associated with a database and/or streaming data. In some examples, a first instruction for enabling execution of a subset of logical operators of the query may be determined. The determination may be based at least in part on the logical operators of the query. Additionally, based at least in part on the first instruction, logical instructions for implementing the query may be generated. Further, the logical instructions may be compiled into machine-readable instructions for implementing only the subset of the logical operators of the query.

Memory sharing across distributed nodes

A method and apparatus are disclosed for enabling nodes in a distributed system to share one or more memory portions. A home node makes a portion of its main memory available for sharing, and one or more sharer nodes mirrors that shared portion of the home node's main memory in its own main memory. To maintain memory coherency, a memory coherence protocol is implemented. Under this protocol, load and store instructions that target the mirrored memory portion of a sharer node are trapped, and store instructions that target the shared memory portion of a home node are trapped. With this protocol, valid data is obtained from the home node and updates are propagated to the home node. Thus, no “dirty” data is transferred between sharer nodes. As a result, the failure of one node will not cause the failure of another node or the failure of the entire system.

RECURSIVE FUNCTIONALITY IN RELATIONAL DATABASE SYSTEMS

A method for execution by a query processing system includes receiving a query expression that includes a call to a computing window function. The computing window function is executed in accordance with execution of the query expression against a database. Execution of the query expression includes accessing an ordered set of rows of the database indicated in the call to the computing window function, and applying a recursive definition indicated in the call to the computing window function to each row in the ordered set of rows to generate output for each row in the ordered set of rows. A query resultant for the query expression is generated based on the output for each row in the ordered set of rows. 1. A method for execution by a query processing system , comprising:receiving a query expression that includes a call to a computing window function; accessing an ordered set of rows of the database indicated in the call to the computing window function;', 'applying a recursive definition indicated in the call to the computing window function to each row in the ordered set of rows to generate output for each row in the ordered set of rows; and', 'generating a query resultant for the query expression based on the output for each row in the ordered set of rows., 'executing the computing window function in accordance with execution of the query expression against a database by2. The method of claim 1 , wherein the query expression includes query language syntax for a window definition indicating the ordered set of rows claim 1 , and wherein the query expression further includes query language syntax indicating an ordering of the ordered set of rows.3. The method of claim 2 , wherein the query language syntax is in accordance with Structured Query Language (SQL) syntax.4. The method of claim 1 , wherein the query expression includes at least one of a plurality of Structured Query Language (SQL) reserved keywords in accordance with SQL syntax claim 1 , and wherein the ...

DATA SEARCHING SYSTEM AND METHOD

The present invention relates to a deep learning system suitable for searching data sources for specific content. In particular, the present invention relates to an unconventional machine-implemented process, leveraging a machine learning algorithm, to provide a technology that searches data sources and recognizes feature data associated with one or more predetermined classifiers. 17.-. (canceled)8. A method for automatically updating machine learning training sets and classifier , the method comprising:storing a machine learning algorithm and machine learning training sets, the machine learning algorithm configured to reference the machine learning training sets;a processor effectuating a recognition agent over one or more data sources;a processor effectuating an extractor that collects feature data from the one or more data sources of the at least one data source based on one or more predetermined classifiers;an input device receiving specific criteria;a processor effectuating an updater that dynamically updates the machine learning training sets and classifier dictionary based on the collected feature data and the specific criteria, enabling the machine learning algorithm to identify at least one feature data including the specific criteria from the collected feature data; andan output device transmitting the at least one feature data including the identified specific criteria;wherein the machine learning algorithm, the machine learning training sets, and the classifier dictionary, are continuously updated to improve performance of the machine learning algorithm over time.9. The method of claim 8 , wherein the effectuating over at least one data source comprises the recognition agent denoting analytics in the data to aggregate feature data.10. The method of claim 9 , further comprising utilizing the machine learning algorithm to search the aggregated feature data for the at least one feature data that is associated with the predetermined classifiers and the ...

SYSTEMS AND METHODS FOR FILTERING ELECTRONIC ACTIVITIES BY PARSING CURRENT AND HISTORICAL ELECTRONIC ACTIVITIES

The present disclosure relates to systems and methods for filtering electronic activities. The method includes identifying an electronic activity. The method includes parsing the electronic activity to identify one or more electronic accounts in the electronic activity. The method includes determining, responsive to parsing the electronic activity, that the electronic activity is associated with an electronic account of the one or more electronic accounts. The method includes selecting, based on the electronic account, one or more filtering policies associated with the data source provider to apply to the electronic activity. The method includes determining, by applying the selected one or more filtering policies to the electronic activity, to restrict the electronic activity from further processing based on the electronic activity satisfying at least one of the selected one or more filtering policies. The method includes restricting, the electronic activity from further processing. 1. A method comprising:identifying, by one or more processors, an electronic activity associated with a data source provider;parsing, by the one or more processors, the electronic activity to identify one or more electronic accounts in the electronic activity;determining, by the one or more processors, responsive to parsing the electronic activity, that the electronic activity is associated with an electronic account of the one or more electronic accounts, the electronic account corresponding to the data source provider;selecting, by the one or more processors based on the electronic account, one or more filtering policies associated with the data source provider to apply to the electronic activity, the selected one or more filtering policies including at least one of i) a keyword policy configured to restrict electronic activities including a predetermined keyword; ii) a regex pattern policy configured to restrict electronic activities including one or more character strings that match ...

Join query execution method and device, and storage medium

A join query execution method and device include a processor and storage medium. An external data source connected to a data management system by a table function is searched, regarding and joining processing against a join query regarding a search result of the table function and a table managed by the data management system, the joining processing is executed on the search result by a first joining method; a join query execution device determines whether the first joining method is appropriate, by using information acquired by search processing on the external data source; and when it is determined that the first joining method is not appropriate, the join query execution device recognizes a speed of a second joining method to be higher than that of the first joining method and switches the joining method for the joining processing to the second joining method.

Digital user interface with item selection

A computer-implemented method of item selection that includes presenting multiple first items that are user selectable in a user interface that each include at least one data field. The method further includes obtaining a user selection of at least one of the first items and generating a rule using a data value from the at least one data field of the user selected item. The method may further include presenting the rule in the user interface. The rule may be user selectable such that deselection of the rule prevents use of the rules in an automatic selection of items among the first items. The method may further include determining an item selection set using the rule. The item selection set may include multiple second items that are a subset of the first items. The second items may include the user selected item and other items of the first items.

QUOTA MANAGEMENT IN A DATASET MANAGEMENT SYSTEM

Systems and methods for controlling access to a dataset management system using quotas are provided. For example, a request to perform an action in a dataset management system may be obtained from an entity, and a quota record associated with the entity may be selected. Further, it may be determined if the entity has sufficient quota to perform the action. In some examples, when the entity has sufficient quota to perform the action, the request may be allowed. In some examples, when the entity has insufficient quota to perform the action, the request may be denied. 120-. (canceled)21. A system for controlling access to a dataset management system using quotas , the system comprising: receive from an external entity a request to perform an action involving accessing information stored in a dataset management system, the information including a record of annotations;', 'based on an identity of the external entity, select a quota record associated with the external entity of a plurality of quota records;', 'based on the selected quota record, determine whether the external entity has sufficient quota to perform the action;', 'select at least one substitute record of annotations of a plurality of alternative records of annotations based on a similarity between the record of annotations and the at least one substitute record of annotations; and', 'based on determining that the external entity has an insufficient quota, provide an indication to the external entity, the indication including a suggestion of the at least one substitute record of annotations., 'at least one processor configured to22. The system of claim 21 , wherein performing the action comprises training a machine learning algorithm using the information from the dataset management system.23. The system of claim 21 , determining whether the external entity has sufficient quota to perform the action comprises:estimating at least one resource requirement associated with performing the action using the ...

DYNAMIC EXECUTION OF PARAMETERIZED APPLICATIONS FOR THE PROCESSING OF KEYED NETWORK DATA STREAMS

A method is described for processing keyed data items that are each associated with a value of a key, the keyed data items being from a plurality of distinct data streams, the processing including collecting the keyed data items, determining, based on contents of at least one of the keyed data items, satisfaction of one or more specified conditions for execution of one or more actions and causing execution of at least one of the one or more actions responsive to the determining. 1. (canceled)2. A method performed by a data processing system for segmenting one or more data items associated with one or more respective key values and one or more respective attributes by executing a computer program for processing the one or more data items associated with the one or more respective key values and the one or more respective attributes , and , for a particular key value , selecting , from among one or more rules , a particular rule to be applied , if the computer program is in a particular state with respect to the particular key value and the one or more data items are associated with one or more attributes of the particular rule , the method including:identifying a computer program that processes one or more data items, with the one or more data items being associated with one or more key values and one or more attributes;executing the computer program to process the one or more data items, wherein one or more states of the computer program are maintained for the one or more key values, the executing including:for one or more data items associated with a particular key value, identifying first and second states of the computer program, with each of the first and second states being associated with the particular key value;based on determining that the computer program is in the first state, with respect to the particular key value, identifying one or more rules;wherein a rule specifies one or more attributes and further specifies one or more actions to be performed ...

PARALLEL MATERIALISATION OF A SET OF LOGICAL RULES ON A LOGICAL DATABASE

A computer-implemented method of providing a materialisation of a set of logical rules on a logical database comprising a set of logical facts. Each of a plurality of parallel processing threads performs the following steps. A logical fact that has not previously been received by any thread is received by the thread. By applying any of the logical rules to the received logical fact and/or any of the logical facts received by any thread prior to the received logical fact, the thread determines whether a new logical fact is implied, and if so the new logical fact is added to the set of logical facts. The thread then repeats the above steps on any further logical facts in the set of logical facts that have not previously been received by any thread. When all threads are waiting for a new logical fact, the method has completed. 139-. (canceled)40. A computer-implemented method , performed by a computer system , the computer system including i) a logical database including a set of logical facts and at least one index that includes a pointer for locating a logical fact in the set of logical facts , and ii) a plurality of parallel processing threads , wherein each of the plurality of parallel processing threads has an associated reserved area in the logical database for adding a new logical fact to the set of logical facts , the method including updating , by the computer system , the logical database , receiving a new logical fact to be added to the logical database;', "adding the new logical fact only to the thread's associated reserved area in the logical database without locking the associated reserved areas of other parallel processing threads;", 'performing an index update process to update the at least one index with the new logical fact without locking the at least one index;, 'wherein one or more threads of the plurality of parallel processing threads of the computer system perform steps comprising 'in a case where the status flag indicates that the at least one ...

INFORMATION PROCESSING APPARATUS, INFORMATION PROCESSING METHOD, AND INFORMATION PROCESSING SYSTEM

According to an embodiment of the present invention, a probability to achieve a purpose is increased in a case where information is intermittently allocated to a plurality of elements. An information processing apparatus according to an embodiment of the present invention is an information processing apparatus that allocates information to respective elements included in a first set and includes an allocator. The allocator uses a first subset and a second subset, and allocates the information to at least one element included in the first subset. The first subset is constituted by elements which are included in the first set and in which information allocation may be performed at a present time. The second subset is constituted by elements which are included in the first set and in which the information allocation is not performed at the present time. 1. An information processing apparatus that allocates information to respective elements included in a first set , the information processing apparatus comprising:an allocator configured to allocate the information to at least one of the element included in a first subset by using the first subset and a second subset, whereinthe first subset is constituted by elements which are included in the first set and in which information allocation may be performed at a present time, andthe second subset is constituted by elements which are included in the first set and in which the information allocation is not performed at the present time.2. The information processing apparatus according to claim 1 , performs a first calculation for calculating candidates of the information to be allocated to the respective elements included in the first subset for each of a plurality of rules,', 'performs a second calculation for calculating candidates of the information to be allocated to the respective elements included in the second subset with respect to a plurality of results of the first calculation, and', 'decides the information to be ...

SUPPORTING JOINS IN DIRECTED ACYCLIC GRAPH KNOWLEDGE BASES

A method, apparatus, and non-transitory computer readable medium for performing joins on data from hierarchical databases are described. The method, apparatus, and non-transitory computer readable medium may provide for receiving one or more search results from each of a plurality of hierarchical databases, identifying one or more matching fields from each of the search results, joining the search results according to a set of rules for processing related fields to the matching fields, wherein the related fields comprise sibling fields, neighbor fields, ancestor fields, descendant fields, or any combination thereof, and generating one or more combined search results based on the joining. 1. A method for performing data joins , comprising:receiving one or more search results from each of a plurality of hierarchical databases;identifying one or more matching fields from each of the search results;joining the search results according to a set of rules for processing related fields to the matching fields, wherein the related fields comprise sibling fields, neighbor fields, ancestor fields, descendant fields, or any combination thereof; andgenerating one or more combined search results based on the joining.2. The method of claim 1 , further comprising:identifying a query for each of the hierarchical databases; andtransmitting the corresponding query to each of the hierarchical databases, wherein the search results from each of the hierarchical databases are received in response to the query.3. The method of claim 2 , wherein;the query for each of the hierarchical databases comprises a same query.4. The method of claim 1 , further comprising;identifying a field name synonym list comprising one or more field name equivalence classes; anddetermining that a field name of the one or more matching fields from each of the search results comprises an equivalent field name based on the field name synonym list.5. The method of claim 4 , further comprising:generating the field name ...

DYNAMIC TRANSACTION COALESCING

The disclosed computer-implemented method includes establishing a coalescing service configured to combine queries received at the coalescing service. The method further includes instantiating, within the coalescing service, multiple execution windows to which the received queries are to be assigned, where each execution window has an assigned deadline within which to execute. The method also includes analyzing a first query among the received queries to identify characteristics of the first query. The method then includes assigning the first query to a first execution window among the execution windows according to the identified characteristics. Then, upon detecting the occurrence of a specified trigger for at least one of the queries in the first execution window, the method includes executing those queries, including the first query, that are assigned to the first execution window. Various other methods, systems, and computer-readable media are also disclosed. 1. A computer-implemented method comprising:establishing a coalescing service configured to combine queries received at the coalescing service;instantiating, within the coalescing service, a plurality of execution windows to which the received queries are to be assigned, each execution window having an assigned deadline within which to execute;analyzing at least a first query among the received queries to identify one or more characteristics of the first query;assigning the first query to a first execution window among the plurality of execution windows according to the identified characteristics; andupon detecting the occurrence of a specified trigger for at least one of the queries in the first execution window, executing those queries, including the first query, that are assigned to the first execution window.2. The computer-implemented method of claim 1 , wherein each of the queries that were assigned to the first execution window before the occurrence of the specified trigger are executed during ...

AUTOMATIC COMPUTATION OF FEATURES FROM A DATA STREAM

Implementations described herein relate to methods, systems, and computer-readable media to automatically compute features from a data stream. In some implementations, a method includes receiving a data stream comprising a plurality of data fields. The method further includes storing at least three copies of data in the data stream in a database, wherein each copy is stored on a corresponding hardware storage device. The method further includes performing, one or more rollup operations on a first copy of the data to compute one or more aggregate features and storing the one or more aggregate features on each of the hardware storage devices. The method further includes receiving a request for a particular aggregate feature and providing the particular aggregate feature in response to the request. 1. A computer-implemented method comprising:receiving, by an ingestion server, a data stream comprising a plurality of data fields;storing, by the ingestion server, at least three copies of data in the data stream in a database, wherein each copy is stored on a corresponding hardware storage device;performing, by a rollup server, one or more rollup operations on a first copy of the data to compute one or more aggregate features;storing, by the rollup server, the one or more aggregate features on each of the hardware storage devices;receiving, by an analytics server, a request for a particular aggregate feature; andproviding, by the analytics server and in response to the request, the particular aggregate feature.2. The computer-implemented method of claim 1 , wherein the ingestion server comprises a plurality of servers in a cluster claim 1 , and wherein a number of the plurality of servers is based on a size of the data stream claim 1 , such that the storing is completed within a storage threshold time from receipt of the data stream.3. The computer-implemented method of claim 1 , wherein the one or more rollup operations are performed such that the one or more aggregate ...

REUSABLE TRANSFORMATION MECHANISM TO ALLOW MAPPINGS BETWEEN INCOMPATIBLE DATA TYPES

In one implementation, a data transformation tool is configured to allow a user to specify how types of data should be mapped to another type of data in a reusable fashion. In one implementation, the data transformation tool analyzes data selected for input, the desired data output, and determines a list of potential mappings that may be used between the compliant and non-compliant systems. The mappings may be generated through transformational expressions and defined as a data wrapper. Once the definition of the one or more mappings are defined, data transformation tool generates one or more files allowing the execution of the transformation of the input data to the output data during an execution phase, such as runtime. 1. A non-transitory computer readable medium including one or more instructions executable by one or more processors for:receiving source data used with a first system to transform to target data used with a second system;analyzing the source data, the first system, and the second system to determine incompatibility of the source data with the second system;analyzing the source data and the target data to determine a plurality of different data mapping functions each configured to map the source data to the second system based on a data mapping criteria specific to a selected data mapping function;receiving a selected data mapping function from the plurality of different data mapping functions, wherein the selected data mapping function includes the data mapping criteria specific to the selected data mapping function;employing the selected data mapping function to generate transformation representation files having a plurality of nodes, wherein at least one node of the plurality of nodes is a proxy node; andconfiguring the proxy node to adapt to at least some mapping errors that occur during mapping of the source data to the second system using the selected data mapping function.2. The non-transitory computer readable medium of claim 1 , wherein ...

TIME SENSITIVE DATA STORE

In an example embodiment, a solution is provided that produces wide tables containing features for machine learned models. These wide tables are made available for model training for multiple models and/or groups. These wide tables may be served on a serving database for fast access for application program interface (API) serving and lightweight access during interactive development. The solution decreases the time needed to add a new feature from several days to a couple of hours by enabling experimentation. 1. A method to store time sensitive data in a database , the method comprising:obtaining time sensitive data;storing the time sensitive data in a wide table in the database; anddefining a separate interface to each of one or more client applications, each of the one or more interfaces pointing to the time sensitive data in the wide table and including a feature layer, the feature layer defining joins between pieces of time sensitive data in the wide table, the defined joins including joins for data pertaining to one or more subject real-estate properties and joins for data pertaining to one or more comparable real-estate properties.2. The method of claim 1 , wherein at least one of the one or more client applications is a machine learning algorithm used to train a machine learned model on the time sensitive data and the corresponding feature layer in the corresponding interface defines the features used in the training.3. The method of claim 1 , wherein the time sensitive data includes comparables about real-estate properties.4. The method of claim 3 , wherein the comparables are stored in a column in the wide table.5. The method of claim 3 , wherein the comparables are stored in a flat table separate from the wide table.6. The method of claim 1 , wherein the wide table contains features precomputed for all possible comparables for all times claim 1 , based solely on distance from a hypothetical subject property claim 1 , for each possible hypothetical subject ...

Ingredient based nutritional information

Nutritional information of a recipe is gathered to determine a nutritional value table of a food recipe. A computing device may extract and analyze unstructured text of a food recipe to obtain a plurality of ingredients and a quantity of the plurality of ingredients. The computing device may access dietary preferences of a user. The nutritional information of the food recipe may be calculated using the nutritional value of each of the ingredients and complied into a nutritional value table. The recipe may be determined if the recipe corresponds with the dietary preferences of the user. If a recipe does not correspond with the dietary preferences, then an ingredient causes the recipe to not correspond with the dietary preferences is removed from the recipe creating an altered recipe. The nutritional value table of the altered recipe is displayed to the user.

Ingredient based nutritional information

Nutritional information of a recipe is gathered to determine a nutritional value table of a food recipe. A computing device may extract and analyze unstructured text of a food recipe to obtain a plurality of ingredients and a quantity of the plurality of ingredients. The computing device may access dietary preferences of a user. The nutritional information of the food recipe may be calculated using the nutritional value of each of the ingredients and complied into a nutritional value table. The recipe may be determined if the recipe corresponds with the dietary preferences of the user. If a recipe does not correspond with the dietary preferences, then an ingredient causes the recipe to not correspond with the dietary preferences is removed from the recipe creating an altered recipe. The nutritional value table of the altered recipe is displayed to the user.

System performance logging of complex remote query processor query operations

Described are methods, systems and computer readable media for performance logging of complex query operations.

Methods and Systems for Processing Geospatial Data

A computer-implemented method and system for generating partition polygons across a geographic area, wherein a concave hull of the reference network, usually a network of linear features such as roads, and the spatial data objects to be processed is first generated, the resulting concave hull thereby providing the boundary geometry for the final partition scheme. By generating a concave hull from the reference network and the spatial data objects, it is possible to derive a boundary that is based on the spatial position of the reference network and the spatial data objects at the periphery of the geographic area that they cover, that is, the boundary itself connects at least a portion of the end points of the reference network and the spatial data objects. 1. A computer-implemented method of generating partition polygons for a first geospatial dataset , the first geospatial dataset comprising a set of spatial data objects to be processed , the method comprising:identifying a set of linear features in a second geospatial data set;generating a first enclosure representative of an area containing the set of spatial data objects in the first data set and the set of linear features in the second dataset; andgenerating a set of partition polygons for the set of spatial data objects in dependence on a boundary of the first enclosure and the set of linear features, wherein a boundary of the first enclosure and the set of linear features define vertices of the partition polygons.2. A method according to claim 1 , wherein the first enclosure is a concave hull.3. A method according to claim 1 , wherein the boundary of the first enclosure comprises a plurality of edge lengths connecting one or more of: a plurality of end points of the linear features and a portion of the spatial data objects.4. A method according to claim 1 , wherein the vertices of the partition polygons are determined based on a plurality of intersections between the boundary of the first enclosure and the ...

INTELLIGENCE SITUATIONAL AWARENESS

An intelligent situational awareness framework may be provided, which may facilitate ingesting the real-time data, persisting at least some of the real-time data on a storage device, analyzing the real-time data to derive at least one insight, and generating an output associated with the at least one insight for real-time visualization. 1. A system comprising:at least one hardware processor; anda storage device coupled with the at least one hardware process; receive real-time data from a plurality of data sources;', 'ingest the real-time data;', 'persist at least some of the real-time data on the storage device;', 'analyze the real-time data to derive at least one insight; and', 'generate an output associated with the at least one insight for real-time visualization., 'the at least one hardware processor operable to at least2. The system of claim 1 , wherein the real-time data comprises at least real-time sensor data.3. The system of claim 1 , wherein the at least one hardware processor ingests the data by transforming and cleansing the real-time data into a format for analyzing.4. The system of claim 1 , wherein the at least one hardware processor executes an artificial intelligence model to analyze the real-time data.5. The system of claim 1 , wherein the at least one hardware processor trains an artificial intelligence model based on the real-time data.6. The system of claim 1 , wherein the at least one hardware processor interfaces with a plurality of user functionalities.7. The system of claim 6 , wherein the plurality of user functionalities comprises data engineering claim 6 , data science claim 6 , analysis and application development claim 6 , wherein the hardware processor allows for data sharing and collaboration among the user functionalities.8. The system of claim 1 , wherein the at least one hardware processor recommends an action based on the at least one insight.9. A computer program product comprising a computer readable storage medium having ...

STREAMS RUNTIME PROCESSING RATE ADJUSTMENT

A stream of tuples is monitored. The stream of tuples is to be processed by a plurality of processing elements of a stream application that operate on one or more compute nodes, each processing element having one or more stream operators. A processing rate of a first stream operator of the stream application is calculated. The processing rate is based on the number of tuples that are processed by the first stream operator. It is determined that the processing rate of the first stream operator meets a predetermined tuple processing criterion. The processing rate of the first stream operator is adjusted based on the predetermined tuple processing criterion. 1. A method for processing streams of tuples , the method comprising:monitoring a stream of tuples to be processed by a plurality of processing elements of a stream application that operate on one or more compute nodes, each processing element having one or more stream operators;calculating a processing rate of a first stream operator of the stream application, the processing rate is based on the number of tuples that are processed by the first stream operator;determining that the processing rate of the first stream operator meets a predetermined tuple processing criterion; andadjusting, based on the predetermined tuple processing criterion, the processing rate of the first stream operator.2. The method of claim 1 , wherein the processing rate is based on a filter statement that defines one or more conditions for tuples to be processed or not processed by the first stream operator.3. The method of claim 2 , wherein the filter statement is located in a definition file.4. The method of claim 3 , wherein the definition file is located outside of the first stream operator.5. The method of claim 2 , wherein adjusting the processing rate comprises modifying the filter statement by updating the one or more conditions the filter statement.6. The method of claim 5 , wherein:the calculating of the processing rate of the ...

Stream Computing Method, Apparatus, and System

A stream computing method performed by a manager node includes obtaining input channel description information, a structured query language (SQL) statement, and output channel description information, dynamically generating a data flow diagram according to the input channel description information, the SQL statement, and the output channel description information, and controlling, according to the data flow diagram, a worker node to execute a stream computing task. 1. A stream computing method performed by a manager node of a stream computing system , wherein the stream computing method comprises:generating a first data flow diagram according to input channel description information, a structured query language (SQL) statement, and output channel description information, wherein the first data flow diagram comprises a plurality of logical nodes, wherein the input channel description information defines an input channel, wherein the input channel is a first logical channel for inputting an input data stream from a data production system to the first data flow diagram, wherein the output channel description information defines an output channel, and wherein the output channel is a logical channel for outputting an output data stream of the first data flow diagram to a data consumption system;classifying the logical nodes in the first data flow diagram to obtain a plurality of logical node groups;selecting, from a preset operator library, a common operator corresponding to each logical node group of the logical node groups;generating a second data flow diagram according to the common operator, wherein each operator in the second data flow diagram implements functions of one or more logical nodes in a logical node group corresponding to the operator; andcontrolling, according to the second data flow diagram, a worker node of the stream computing system to execute a stream computing task.2. The stream computing method of claim 1 , wherein the first data flow diagram ...

Spool file for optimizing hash join operations in a relational database system

A method for spooling data for use in joining a small table with a large table in a relational database system. The method analyzes a join condition for combining records from the small and large tables, selects qualified rows from the large table, and writes the qualified rows to a spool file. The spool file includes a first partition containing hash values of all bind terms for the join condition; a second partition including a join column with a best selective bind term; and at least one additional partition including additional join columns used in bind terms. The partitions are grouped together within a container row in the spool file, and multiple container rows are written together within a super-container row in the spool file.

Efficient data retrieval based on aggregate characteristics of composite tables

Techniques related to efficient data retrieval based on aggregate characteristics of composite tables are provided. A join zone map includes entries that describe data from a join relationship between a first key column of a first table and a second key column of a second table. The first table includes a dimension column. Each entry of the join zone map corresponds to a respective zone. Each zone includes contiguous data blocks that correspond to one or more second key column values. Each entry also includes a respective dimension value range of one or more dimension column values. Each dimension value range includes a respective maximum dimension value and a respective minimum dimension value. Furthermore, each entry includes a respective anti-join attribute value that indicates whether any of the one or more second key column values in a particular zone are non-null and fail to match any first key column values.

Method and System for Accelerated Stream Processing

Disclosed herein are methods and systems for hardware-accelerating various data processing operations in a rule-based decision-making system such as a business rules engine, an event stream processor, and a complex event stream processor. Preferably, incoming data streams are checked against a plurality of rule conditions. Among the data processing operations that are hardware-accelerated include rule condition check operations, filtering operations, and path merging operations. The rule condition check operations generate rule condition check results for the processed data streams, wherein the rule condition check results are indicative of any rule conditions which have been satisfied by the data streams. The generation of such results with a low degree of latency provides enterprises with the ability to perform timely decision-making based on the data present in received data streams. 1. A system for making a reconfigurable logic device available in a network for loading a processing pipeline thereon for the reconfigurable logic device to apply parallelism when processing streaming data , the streaming data comprising data arranged in a plurality of fields , the system comprising:a reconfigurable logic device that is addressable within a network, the reconfigurable logic device arranged for configuration in response to a command over the network so that a processing pipeline for receiving and processing streaming data is loadable thereon;the loadable processing pipeline including a plurality of parallel paths, each of a plurality of the parallel paths including pipelined logic for performing different processing operations on the streaming data, and wherein each of a plurality of the parallel paths includes field selection logic that filters which fields of the streaming data that downstream pipelined logic in that parallel path will process, wherein a plurality of the parallel paths include field selection logic that filter for different fields of the streaming ...

Logical partitions via header-based partition filtering

Techniques for storing and processing large volumes of data by increasing the number of partitions that customers can allocate and use for their streams while minimizing the physical footprint of the partitions is disclosed. In certain examples, the stream partitions are created as different logical partitions on a physical partition in a physical storage cluster of a distributed message streaming system. The disclosed system provides an improved process for reading messages stored in different logical partitions of a physical partition by identifying partition chunks in a physical partition that comprise subsets of messages associated with a logical partition and filtering out partition chunks that do not contain messages associated with the logical partition. The disclosed system reduces the amount of the physical partition that needs to be read to retrieve messages stored in different logical partitions thereby improving the overall performance of the distributed message streaming system.

INTEGRATED INTELLIGENT BUILDING MANAGEMENT SYSTEM AND MANAGEMENT METHOD THEREOF

An integrated intelligent building management system and a management method thereof are disclosed. The system includes a first cloud management system having a project management database, an intelligent building system managed by the first cloud management system, and a second cloud management system connecting with the first cloud-management system through a transforming interface. The management method includes the following steps. The first cloud management system receives the corresponding data of the building controlled by the intelligent building system and stores that in the project management database. The second cloud management system is authorized to connect and access the project management database. An access action from the second cloud management system to the project management database is accepted. A format transformation to data corresponding to the access action is performed. The second cloud management system stores the transformed data to a demand database therein. 1. An integrated intelligent building management system , comprising:a first cloud management system, comprising a first cloud configuration platform and a project management database comprising a first transforming interface and a data transforming service module;an intelligent building system, connected with and managed by the first cloud management system, and a corresponding data of a building controlled by the intelligent building system being stored in the project management database; anda second cloud management system, comprising a second cloud configuration platform and a demand database comprising a second transforming interface;wherein the first cloud management system is connected with the second transforming interface of the second cloud management system through the first transforming interface and authorizes the second cloud management system to connect and access the project management database,the first cloud management system performs a format transformation to ...

METHOD AND SYSTEM FOR FILTERING DATABASE QUERIES

A method and a system for dynamically scanning, filtering, and blocking harmful database queries that would otherwise consume significant resources and adversely impact overall system performance are provided. The method includes: receiving a user request for data from a database, the first request including a query; applying database access rules to the query in order to determine whether the query is potentially harmful; when the query is determined as not potentially harmful, forwarding the request to a server configured to respond to the request; and when the first query is determined as being potentially harmful, transmitting a warning message to the user. The database access rules may include a maximum memory consumption rule and a maximum CPU consumption rule. Machine learning techniques are used for adjusting the database access rules based on historical data. 1. A method for controlling access to a database , the method being implemented by at least one processor , the method comprising:receiving, from a user by the at least one processor, a first request for data that is accessible via a database, the first request including a first query;analyzing, by the at least one processor, the first query to determine whether the first query is potentially harmful;when the first query is determined as not potentially harmful, forwarding, by the at least one processor, the first request to a server configured to respond to the first request; andwhen the first query is determined as being potentially harmful, transmitting, to the user by the at least one processor, a warning message.2. The method of claim 1 , wherein the analyzing comprises applying at least one database access rule to the first query.3. The method of claim 2 , wherein the at least one database access rule includes at least one from among a rule that relates to a maximum amount of memory to be consumed by granting the first request and a rule that relates to a maximum amount of central processing unit ...

Registration Method for Broadcast Receiver, Terminal and Storage Medium

A method of registration for a broadcast receiver, a terminal and a non-transitory computer readable storage medium are provided. The method includes: classifying a broadcast sender according to a frequency of a broadcast message sent by the broadcast sender and called by a broadcast recipient, and recording the broadcast sender and a type of the broadcast sender determined in a classification result; querying, after obtaining a registration instruction of the broadcast receiver and according to a registration object in a registration instruction, the type of the broadcast sender corresponding to the registration object from the classification result; and processing, according to the type of the broadcast sender, the registration of the broadcast receiver. 1. A method for registering a broadcast receiver , comprising:classifying a broadcast sender according to a frequency of a broadcast message sent by the broadcast sender and called by a broadcast recipient, and recording the broadcast sender and a type of the broadcast sender determined in a classification result;querying, after obtaining a registration instruction of the broadcast receiver and according to a registration object in a registration instruction, the type of the broadcast sender corresponding to the registration object from the classification result; andprocessing, according to the type of the broadcast sender, the registration of the broadcast receiver.2. The method according to claim 1 , wherein classifying the broadcast sender according to the frequency of the broadcast message sent by the broadcast sender and called by the broadcast recipient comprises:determining the type of the broadcast sender as a high-risk type, when the frequency of a broadcast message sent by the broadcast sender and called by a broadcast recipient is greater than a first preset frequency;determining the type of the broadcast sender as a risk type, when the frequency of a broadcast message sent by the broadcast sender and ...

Object relational mapping with a single database query

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating an object graph from records of a relational database. One of the methods includes receiving a request to populate a software object with values from a plurality of tables represented in a graph, the plurality of tables stored in a relational database maintained by a database management system, each table comprising one or more respective tuples of values; generating a single query statement that, when executed by the database management system, causes the database management system to retrieve the values from the plurality of tables represented in the graph as a result set; querying the relational database using the single query statement to obtain the result set; and populating the software object with one or more values of the result set.

LOAD BALANCING IN STREAMS PARALLEL REGIONS

In response to receiving a flow of data, one or more operators in a parallel region included in a distributed data processing system for stream processing the flow of data are initialized. In response to identifying a change to the flow of data or a change to the distributed data processing system, at least one of the flow of data and a size of the parallel region included in the distributed data processing system are dynamically adjusted. The flow of data is processed via the dynamically adjusted one of the flow of data and the parallel region included in the distributed data processing system. 1. A method , the method comprising:responsive to determining a rate of a flow of data, initializing, by one or more computer processors, one or more operators in a parallel region included in a distributed data processing system for stream processing of the flow of data based on the determined rate;responsive to identifying a change in the flow of data or a change in the distributed data processing system, dynamically adjusting, by one or more computer processors, the flow of data in the distributed data processing system; andprocessing, by one or more computer processors, the flow of data via the dynamically adjusted flow of data in the distributed data processing system.2. (canceled)3. The method of claim 1 , wherein the step of dynamically adjusting claim 1 , by one or more computer processors claim 1 , the flow of data in the distributed data processing system claim 1 , comprises:dynamically adjusting, by one or more computer processors, the flow of data between the one or more operators in the parallel region to balance a load of the flow of data.4. The method of claim 1 , wherein the step of dynamically adjusting claim 1 , by one or more computer processors claim 1 , the flow of data in the distributed data processing system claim 1 , comprises:achieving, by one or more computer processors, a highest data rate to the parallel region.5. The method of claim 1 , wherein ...

MULTI-USER CROSS-DEVICE TRACKING

Embodiments collect, from an event stream having both client-side and server-side information from multiple tracking sources, user interaction records for a web application and supporting systems. The records are stored in a data store, and are retroactively correlated with conclusive user identifiers and, if applicable, with areas of interest. Attribution of interaction records to conclusive identifiers and areas of interest is based on links that have been established between identifiers. Established identifier links may be subject to one or more link removal criteria. If link removal criteria is satisfied for an established link, the link is broken. Once a link is broken, interaction records, found in the data store, that are associated with the inconclusive identifier of the broken link are no longer associated with the conclusive identifier of the broken link. Each interaction record that is associated with a conclusive identifier is assigned a confidence metric for the association. 1. A computer-executed method comprising:determining that a particular information item, of a particular plurality of information items reflecting client-server interactions, is associated with (a) a particular inconclusive identifier, and (b) a conclusive identifier; and identifying a particular set of information items, of the particular plurality of information items, that are associated with the particular inconclusive identifier, and', 'storing, in a data store, attribution data that associates the particular set of information items with the conclusive identifier;, 'responsive to determining that the particular information item is associated with the conclusive identifier and the particular inconclusive identifierwherein the method is performed by one or more computing devices.2. The computer-executed method of claim 1 , further comprising detecting the particular plurality of information items in an event stream populated from commit logs of one or more database management ...

Dynamic domain query and query translation

Embodiments are directed to defining a query expression over a dynamic system model and to transforming a dynamically built query expression into a simplified, canonical form. A computer system receives various user inputs that are to be included as part of a query expression for querying a dynamic system model. The dynamic system model is queryable to determine features provided by a service hosted by a back-end server. The computer system dynamically builds the query expression based on both the received inputs and the available features provided by the service as indicated by the dynamic system model. The query expression also includes metadata structures representing those features of the service which are provided by the service. The computer system also queries across the dynamic system model using the dynamically built query expression to retrieve and provide the service features requested in the received user input.

Weighted similarity estimation in data streams with applications to collaborative filtering and viral marketing

A method estimates similarities in data streams. A data source receives input vectors from users. A sketch feature generator converts the input vectors into sketch feature vectors corresponding to the users, wherein each sketch feature vector represents data and meta-information from each user received in a most recent sample period. A similarity comparator compares each sketch feature vector against other sketch feature vectors to calculate similarity probabilities between the users. A processor running a decision loop determines a prediction result for at least one user based on the similarity probabilities.

General and automatic approach to incrementally computing sliding window aggregates in streaming applications

A method of incrementally computing an aggregate function of a sliding window in a streaming application includes receiving a plurality of data tuples in the sliding window, extracting at least one data tuple from the sliding window, and storing the at least one extracted data tuple in a data structure in a memory. The data structure is a balanced tree and the at least one data tuple is stored in leaf nodes of the balanced tree. The method further includes maintaining at least one intermediate result in at least one internal node of the balanced tree. The at least one intermediate result corresponds to a partial window aggregation. The method further includes generating a final result in the balanced tree based on the at least one intermediate result, and outputting the final result from the balanced tree. The final result corresponds to a final window aggregation.

SEARCH DEVICE, SEARCH METHOD, AND SEARCH PROGRAM

A storage unit () stores a hash table that corresponds to a predetermined field of a packet and has a set of a hash value from a value in the corresponding field and a pointer registered, and an ANY point table that corresponds to the predetermined field and has a pointer registered. A search unit () refers to a hash value calculated from a value in a field of a packet to be searched and to a pointer registered in the hash table and the ANY point table, to perform hash search for a rule corresponding to the packet to be searched. 1. A search apparatus comprising:a storage unit configured to store a hash table corresponding to a predetermined field of a packet, the hash table having a set of a hash value from a value for a corresponding field and a pointer registered, and an ANY point table corresponding to the predetermined field, the ANY point table having a pointer registered; anda search unit configured to refer to (i) a hash value calculated from a value for a field of a packet to be searched and a pointer registered in the hash table and (ii) the ANY point table to perform hash search for a rule corresponding to the packet to be searched.2. The search apparatus according to claim 1 , wherein:the storage unit is configured to store at least one of a pointer pointing a hash table corresponding to a field next to the predetermined field or a pointer pointing an ANY point table corresponding to the field next to the predetermined field, as a pointer registered in the hash table and the ANY point table.3. The search apparatus according to claim 1 , further comprising a creation unit configured to create a hash table based on a value being set as a search condition among the search conditions for the value in each field of the packet to be searched claim 1 , and to create an ANY point table based on the ANY condition being set as the search condition.4. The search apparatus according to claim 3 , wherein:the creation unit is configured to register NULL in the pointer ...

Methods, apparatuses, and systems for ingesting and consuming data utilizing a trading partner manager

Example methods, apparatuses, and systems are disclosed for ingesting and consuming data utilizing a trading partner manager. In one example embodiment, an example computing device receives user input comprising a set of data fabric configurations. The computing device then receives, into a data landing zone and based on the data fabric configurations, a data set from a data source. The computing device then ingests, based on the data fabric configurations, the received data set from the data landing zone into a data lake. The computing device then enriches, based on the data fabric configurations, the ingested data set. The computing device then enables data consumption of the ingested data set by the user.

Method and system for implementing subscription barriers in a distributed computation system

Embodiments of the invention relate to a method for managing subscriptions. The method includes initiating execution of a first subscription, in response to the initiating, obtaining a first subscription barrier associated with a first subscription, making, using the first subscription barrier, a first determination to block execution of a first query request associated with the first subscription, and in response to the first determination, ceasing execution of the first subscription.

Parallel Processing of Data Having Data Dependencies for Accelerating the Launch and Performance of Operating Systems and Other Computing Applications

Representative embodiments are disclosed for a rapid and highly parallel decompression of compressed executable and other files, such as executable files for operating systems and applications, having compressed blocks including run length encoded (“RLE”) data having data-dependent references. An exemplary embodiment includes a plurality of processors or processor cores to identify a start or end of each compressed block; to partially decompress, in parallel, a selected compressed block into independent data, dependent (RLE) data, and linked dependent (RLE) data; to sequence the independent data, dependent (RLE) data, and linked dependent (RLE) data from a plurality of partial decompressions of a plurality of compressed blocks, to obtain data specified by the dependent (RLE) data and linked dependent (RLE) data, and to insert the obtained data into a corresponding location in an uncompressed file. The representative embodiments are also applicable to other types of data processing for applications having data dependencies.

Methods Data Structures Circuits Devices Systems and Functionally Associated Machine Executable Code for Artificial Intelligence Based Situational Assessment

Disclosed herein are methods, structures, circuits, devices, systems and functionally associated computer executable code for artificial intelligence based situational assessment. According to some embodiments, there may be provided a location specific situational assessment system comprising a sensor interface to receive sensing related data from each of one or more location specific sensors directed towards and acquiring information from a respective coverage area of a specific location. 1. A location specific situational assessment system comprising:a sensor interface to receive sensing related data from each of one or more location specific sensors directed towards and acquiring information from a respective coverage area of a specific location;a processing platform including processing resources for running one or more agent applications and an agent application deployment controller, wherein said agent application deployment controller, responsive to location specific information embedded within data received from said one or more sensors, selects from a pool of agent application templates an agent application template corresponding to the location specific information and deployes within said processing platform of said system an agent application based on said selected agent application template; andwherein agent application deployment within said processing platform includes establishing data connectivity between the agent application being deployed and one or more system resources.2. The situational assessment system according to claim 1 , wherein said system resources with which connectivity is established is selected from the group consisting of: (a) a sensor feed; (b) another agent applications running on the computing platform; (c) system data storage; and (d) a neural network.3. The situational assessment system according to claim 2 , wherein access to one or more of said systems resources is secured and establishing connectivity between the agent ...

MERGING DIRECTORY INFORMATION FROM A USER DIRECTORY TO A COMMON DIRECTORY

Merging directory information from a user directory to a common directory includes based on a rule associated with contents of a user directory of a local database, extracting a department indicator from the user directory, mapping the department indicator to a corresponding user record in a common directory of a registry database, and based on the department indicator, marking the user record in the common directory with a department that corresponds to the contents from the user directory with a department for controlling access to that user record in the registry database. 1. A method for merging directory information from a user directory to a common directory , the method comprising , with a computer having a network interface for communicating with a computer network:based on a rule associated with contents of a user directory of a local database, extracting a department indicator from the user directory, wherein the department indicator indicates to which department within a principal organization that a user or a smaller organization belongs;mapping the department indicator to a corresponding user record in a common directory of a registry database;based on the department indicator, marking the user record in the common directory that corresponds to the contents from the user directory with a department; andbased on the marking in the common directory, controlling access to that user record in the registry database.2. The method of claim 1 , wherein the rule comprises a regular expression pattern to extract the department indicator from the user directory.3. The method of claim 1 , wherein a secondary rule is established when a primary rule fails to determine the department.4. The method of claim 3 , wherein the secondary rule utilizes an alternate user directory to determine the department indicator.5. The method of claim 3 , wherein a default value is used when the primary rule and the secondary rule fails to determine the department.6. The method of claim ...

Methods, Systems, and Products for Maintaining Data Consistency in a Stream Warehouse

Methods, systems, and products characterize consistency of data in a stream warehouse. A warehouse table is derived from a continuously received a stream of data. The warehouse table is stored in memory as a plurality of temporal partitions, with each temporal partition storing data within a contiguous range of time. A level of consistency is assigned to each temporal partition in the warehouse table.

SYSTEM AND METHOD FOR ADAPTIVE INFORMATION STORAGE MANAGEMENT

A system and a method for adaptive information storage management is provided. One or more parameters from an incoming data stream is identified based on a set of predetermined rules. The identified parameters correspond to a set of predetermined parameters. A subset of rules is applied, from the set of predetermined rules, on the incoming data stream. The subset of rules represent a series of iterative rules associated with each identified parameter. One or more data storage allocation files are generated that represent results of application of the set of predetermined rules and the subset of rules. The results include at least a storage system type identified from the storage system as optimal for storing the incoming data stream. 1. A computer-implemented method for adaptive information storage management , wherein the method is implemented by a system receiving a data stream from at least one client station , the system is coupled to a storage system , the method comprising:identifying one or more parameters from an incoming data stream based on a set of predetermined rules, wherein the identified parameters correspond to a set of predetermined parameters;applying a subset of rules, from the set of predetermined rules, on the incoming data stream, wherein the subset of rules represent a series of iterative rules associated with each identified parameter; andgenerating one or more data storage allocation files representing results of application of the set of predetermined rules and the subset of rules, wherein the results include at least a storage system type identified from the storage system as optimal for storing the incoming data stream.2. The computer-implemented method of claim 1 , wherein the method comprises generating a system storage allocator file based on the data storage application files claim 1 , wherein the system storage allocator file includes information related to transmission of the incoming data stream to the identified storage system ...

Systems and methods to improve data clustering using a meta-clustering model

Systems and methods for clustering data are disclosed. For example, a system may include one or more memory units storing instructions and one or more processors configured to execute the instructions to perform operations. The operations may include receiving data from a client device and generating preliminary clustered data based on the received data, using a plurality of embedding network layers. The operations may include generating a data map based on the preliminary clustered data using a meta-clustering model. The operations may include determining a number of clusters based on the data map using the meta-clustering model and generating final clustered data based on the number of clusters using the meta-clustering model. The operations may include and transmitting the final clustered data to the client device.

Systems and methods for hyperparameter tuning

A model optimizer is disclosed for managing training of models with automatic hyperparameter tuning. The model optimizer can perform a process including multiple steps. The steps can include receiving a model generation request, retrieving from a model storage a stored model and a stored hyperparameter value for the stored model, and provisioning computing resources with the stored model according to the stored hyperparameter value to generate a first trained model. The steps can further include provisioning the computing resources with the stored model according to a new hyperparameter value to generate a second trained model, determining a satisfaction of a termination condition, storing the second trained model and the new hyperparameter value in the model storage, and providing the second trained model in response to the model generation request.

Systems and methods to identify neural network brittleness based on sample data and seed generation

Systems and methods for determining neural network brittleness are disclosed. For example, the system may include one or more memory units storing instructions and one or more processors configured to execute the instructions to perform operations. The operations may include receiving a modeling request comprising a preliminary model and a dataset. The operations may include determining a preliminary brittleness score of the preliminary model. The operations may include identifying a reference model and determining a reference brittleness score of the reference model. The operations may include comparing the preliminary brittleness score to the reference brittleness score and generating a preferred model based on the comparison. The operations may include providing the preferred model.

ONLINE, INCREMENTAL REAL-TIME LEARNING FOR TAGGING AND LABELING DATA STREAMS FOR DEEP NEURAL NETWORKS AND NEURAL NETWORK APPLICATIONS

Today, artificial neural networks are trained on large sets of manually tagged images. Generally, for better training, the training data should be as large as possible. Unfortunately, manually tagging images is time consuming and susceptible to error, making it difficult to produce the large sets of tagged data used to train artificial neural networks. To address this problem, the inventors have developed a smart tagging utility that uses a feature extraction unit and a fast-learning classifier to learn tags and tag images automatically, reducing the time to tag large sets of data. The feature extraction unit and fast-learning classifiers can be implemented as artificial neural networks that associate a label with features extracted from an image and tag similar features from the image or other images with the same label. Moreover, the smart tagging system can learn from user adjustment to its proposed tagging. This reduces tagging time and errors. 1. A method of tagging a sequence of images , the method comprising:tagging, by a user, a first instance of a representation of an object in a first image in a sequence of images;learning, by at least one processor, the representation of the object tagged by the user in the first image;tagging, by the at least one processor, a second instance of the representation of the object in the sequence of images;performing an adjustment, by the user, of a tag and/or position of the second instance of the representation of the object created by the at least one processor; andtagging, by the at least one processor based on the adjustment, a third instance of the representation of the object in the sequence of images.2. The method of claim 1 , wherein the second instance of the representation of the object is in the first image in the sequence of images.3. The method of claim 1 , wherein the second instance of the representation of the object is in another image in the sequence of images.4. The method of claim 1 , further comprising: ...

Provisioning for smart navigation services

Techniques for provisioning a smart navigation service are presented. The provisioning can be performed by a name owner, by the smart navigation service itself, or by a third-party keyword service. The provisioned information can include an entity name, a keyword, and possibly other data correlated to at least one network locator. The navigation service electronically stores in navigation service persistent memory a rule correlating the entity name, the keyword, and, if used, the other data, to the at least one network locator, such that when the navigation service receives, from a client computer communicatively coupled to the navigation service, command data that includes the entity name, the keyword, and possibly other data, the navigation service responds to the client computer with the at least one network locator.

METHODS AND SYSTEMS FOR FIXED INTERPOLATION ERROR DATA SIMPLIFICATION PROCESSES FOR TELEMATICS

Methods and systems for simplifying data collected from assets are provided. An example method involves receiving a set of simplified data at a server. The simplified set of data is generated by application of a dataset simplification algorithm on raw data obtained from a data source at an asset upon satisfaction of a data logging trigger, wherein the dataset simplification algorithm causes interpolation error within the simplified set of data to be limited by an upper bound that is fixed across the simplified set of data. The method further involves receiving a request for a status of the asset and interpolating a status of the asset based on the simplified set of data in response to the request. 1. A system comprising:an interface layer at an asset to receive raw data from a data source at the asset; determine whether a data logging trigger is satisfied; and', 'when the data logging trigger is satisfied, perform a dataset simplification algorithm on the raw data to generate a simplified set of data in which interpolation error is limited by an upper bound that is fixed across the simplified set of data;, 'a controller at the asset toa communication interface at the asset to transmit the simplified set of data; and receive the simplified set of data; and', 'interpolate a status of the asset based on the simplified set of data in response to a status request., 'a server to2. The system of claim 1 , wherein the server is to indicate the upper bound on interpolation error.3. The system of claim 1 , wherein the server is to provide an indication of the interpolated status of the asset and an indication of the upper bound on interpolation error to an end user device.4. The system of claim 1 , wherein the raw data includes a target set of data that is recorded over time claim 1 , and generating the simplified set of data involves including a point from the target set of data in the simplified set of data if the point is distant claim 1 , along a data dimension of the ...

Detecting data skew in a join operation

Systems, methods, and devices, for managing data skew during a join operation are disclosed. A method includes computing a hash value for a join operation and detecting data skew on a probe side of the join operation at a runtime of the join operation using a lightweight sketch data structure. The method includes identifying a frequent probe-side join key on the probe side of the join operation during a probe phase of the join operation. The method includes identifying a frequent build-side row having a build-side join key corresponding with the frequent probe-side join key. The method includes asynchronously distributing the frequent build-side row to one or more remote servers.

Intelligent remote monitoring method for fire-fighting

Disclosed is an intelligent remote monitoring method for fire-fighting, including: S1, pre-loading a protocol parsing configuration file in JSON format; S2, acquiring fire-fighting status information and operation status information of fire-fighting IoT equipment to obtain a data frame of a binary protocol; S3, parsing the data frame according to the configuration file to obtain a JSON data object, and pushing the parsed data of the fire-fighting IoT equipment to a routing layer through data push technology including kafka technology and storing in a fire-fighting database; S4: presetting different application modules according to different application services, and determining a routing direction of the data of the fire-fighting IoT equipment according to rules including a device type and a device status, and routing the data of the corresponding fire-fighting IoT equipment to the corresponding application module through the routing layer for triggering processing of events including alarms and faults, so as to realize remote monitoring of fire-fighting.

DATA PROVISION METHOD IN A DATA UTILIZATION SYSTEM

To promote data utilization by providing processed data while leaving a trail that is hardly falsified. A data management server B receives an analysis request, executes an analysis program B, and transfers the analysis request to a data processing server allowing second data resulting from processing first data to be saved in storage (). The data management server B further receives the notification of having processed the first data from the data processing server (), generates a transaction indicating the first data having been processed according to the analysis request (), and adds the verified transaction as a record in a processing/usage trail management table as one of distributed shared ledgers (). 1. A data provision method in a data utilization system that is connected to a network and has: a plurality of data management servers including a first data management server for a first user and a second data management server for a second user; and a data processing server ,wherein each of the data management servers maintains a distributed shared ledger;wherein the first data management server executes a data registration program to save first data in storage, generates a first transaction indicating the first data having been saved, and adds the verified first transaction as a record in a metadata management table as one of the distributed shared ledgers;wherein the second data management server transmits an analysis request for processing the first data to the first data management server;wherein the first data management server receives the analysis request, executes an analysis program, transfers the analysis request to the data processing server, allowing second data resulting from processing the first data to be saved in storage, receives notification of having processed the first data from the data processing server, generates a second transaction indicating the first data having been processed according to the analysis request, and adds the verified ...

Resource provisioning systems and methods

A method and apparatus managing a set of processors for a set of queries is described. In an exemplary embodiment, a device receives a set of queries for a data warehouse, the set of queries including one or more queries to be processed by the data warehouse. The device further provisions a set of processors from a first plurality of processors, where the set of processors to process the set of queries, and a set of storage resources to store data for the set of queries. In addition, the device monitors a utilization of the set of processors as the set of processors processes the set of queries. The device additionally updates a number of the processors in the set of processors provisioned based on the utilization/Furthermore, the device processes the set of queries using the updated set of processors.

ACCESSING DATA OF CATALOG OBJECTS

Example systems and methods for cloning catalog objects are described. In one implementation, a method includes creating a copy of a catalog object without copying a data associated with the catalog object by only coping metadata associated with the object. The method further includes modifying, by one or more processors, the data associated with the catalog object independently of the copy of the catalog object. 1. A system comprising:a memory to store an original catalog object associated with a dataset, and create a duplicate catalog object of the original catalog object by copying metadata associated with the dataset without copying the dataset;', 'determine, based on the metadata, whether the dataset needs to be accessed without accessing the dataset; and', 'access, based on the duplicate catalog object, the dataset associated with the original catalog object responsive to determining that the dataset needs to be accessed., 'one or more processors, operatively coupled to the memory, the one or more processors to2. The system of claim 1 , wherein the dataset is stored in files claim 1 , and wherein the one or more processors are further to:execute data access requests directed to the dataset stored in the files by reading the duplicate catalog object of the original catalog object when the dataset is being recreated; andadd additional files to either of the original catalog object or the duplicate catalog object of the original catalog object independently of another.3. The system of claim 1 , wherein the duplicate catalog object comprises a duplicate hierarchy of one or more generations of children.4. The system of claim 3 , wherein to copy the metadata the one or more processors are further to copy an inventory of the dataset.5. The system of claim 3 , wherein to copy the metadata the one or more processors are further to copy information regarding the dataset that enables identification of the dataset without requiring access to the dataset.6. The system of ...

METHODS AND SYSTEMS FOR INTERACTIVE CONTENT CREATION

A system is provided for personalizing an interactive content. The system comprises: a content creation component configured to create one or more content items for a document, and at least one content item is associated with a set of rules defining a link between the at least one content item and another document or a rule for assembling the at least one content into a personalized document; and a content personalization component configured to: (i) receive an instruction for personalizing a given document; (ii) assemble a plurality of content items to generate a personalized document, and the plurality of content items are pulled from the given document and at least another document based at least in part on a set of rules associated with at least one content item of the given document; and (iii) render the personalized document using a brand style associated with the given document. 1. A system for automatically assembling an interactive document comprising:a server in communication with a plurality of user devices over a network, wherein the server comprises: a memory for storing a set of software instructions, and one or more processors configured to execute the set of software instructions to:(a) embed a first set of rules into a first modular item of a first document;(b) receive one or more parameters for personalizing the first document;(c) pull a second modular item from a second document based at least in part on a value of the one or more parameters and the first set of rules; and(d) assemble the second modular item with the first document to generate a derivative document.2. The system of claim 1 , wherein the first modular item is stored in a structured data that defines the first set of rules.3. The system of claim 1 , wherein the first set of rules define a link between the first modular item and another document or a rule for assembling the first modular item into a derivative document.4. The system of claim 1 , wherein the value of the one or more ...

Database diagnostics interface system

A database diagnostics system with an interface system that may be used to define, monitor, and deploy database diagnostics tools is presented. The interface system presents a user with a user interface for defining the parameters, behaviors, and schedules of database diagnostic tools. The diagnostic tools execute on a target database recording database parameters and state information. The interface system may present the user with a graphical user interface for assembling diagnostic tools at least partially from a predefined set of reusable modules and scripts.

System and method for aiding a user decision regarding time-sensitive data elements

Provided is a system and method for aiding a user in deciding upon an appropriate action to take regarding a plurality of time-sensitive data elements, the time-sensitive data elements stored in a first database, based upon stored further information data elements stored in a second database. The user is provided with a report which comprises links between the time-sensitive data elements and the stored further information data elements.

COMPUTER METHODS AND SYSTEMS FOR IDENTIFYING PARTS AND INVENTORY

Disclosed are methods, systems, and non-transitory computer-readable medium for identifying missing parts and stranded inventory. For instance, the method may include: obtaining parts data and work in progress (WIP) data associated with one or more production facilities, the parts data indicating one or more parts in an inventory to be used to produce one or more products, the WIP data indicating one or more WIP products currently in production; analyzing the parts data and the WIP data to determine missing parts and/or on-hand parts for the one or more WIP products; and performing at least one action for the missing parts and/or the on-hand parts. 1. A method for identifying missing parts and stranded inventory , comprising:obtaining parts data and work in progress (WIP) data associated with one or more production facilities, the parts data indicating one or more parts in an inventory to be used to produce one or more products, the WIP data indicating one or more WIP products currently in production;analyzing the parts data and the WIP data to determine missing parts and/or on-hand parts for the one or more WIP products; andperforming at least one action for the missing parts and/or the on-hand parts.2. The method of claim 1 , further comprising claim 1 , before the analyzing the parts data and the WIP data:determining whether a trigger condition is satisfied; andin response to determining the trigger condition is satisfied, analyzing the parts data and the WIP data to determine the missing parts and/or the on-hand parts for the one or more WIP products.3. The method of claim 1 , wherein the parts data includes:a part ID for each part used in the one or more products produced in the one or more production facilities,a count for each part ID that corresponds to a number of parts in the inventory for that part ID,location information for each part in the inventory, and/orexpected cost for each part.4. The method of claim 3 , wherein the WIP data includes the one or ...

Content filtering for personal productivity applications

A set of context profiles containing profile content rules for filtering which content within a set of personal productivity applications (such as e-mail, chat, calendar, and contacts applications) is accessible by a user. Content rules support filtering content by, for example, content originator, and/or subject matter. The context profiles are centrally maintained for control of integrated, non-integrated, and/or independently administered personal productivity applications.