БЕЗОПАСНОСТЬ В ПРИЛОЖЕНИЯХ СИНХРОНИЗАЦИИ РАВНОПРАВНЫХ УЗЛОВ

Изобретение относится к области синхронизации данных в системах хранения данных. Техническим результатом является упрощение синхронизации данных в системах хранения данных. Способ создания связи синхронизации в защищенном сообществе с синхронизацией между равноправными узлами содержит этапы, на которых принимают ввод пользователя на первом вычислительном устройстве, который уточняет, что первая папка на первом вычислительном устройстве должна быть синхронизирована со второй папкой на втором вычислительном устройстве; в ответ на ввод, первое вычислительное устройство создает приглашение, которое включает в себя информацию для создания связи синхронизации между первой и второй папками, причем информация включает в себя список членов, который задает права доступа к первой папке для каждого пользователя в синхронизированном сообществе, так что список членов включает в себя, по меньшей мере, одного пользователя другого вычислительного устройства в синхронизированном сообществе в дополнение к ...

СПОСОБ И СИСТЕМА ДЛЯ СИНХРОНИЗАЦИИ МНОЖЕСТВЕННЫХ ПОЛЬЗОВАТЕЛЬСКИХ РЕВИЗИЙ СОВМЕСТНО ИСПОЛЬЗУЕМОГО ОБЪЕКТА

Изобретение относится к системе связи и предназначено для синхронизации множественных пользовательских ревизий совместно используемого объекта. Технический результат - повышение точности синхронизации. Множество разных пользователей могут одновременно осуществлять доступ к одному и тому же совместно используемому объекту, ревизовать и обновлять его через несколько разных транспортных механизмов. Ревизии совместно используемого объекта автоматически синхронизируются, что позволяет всем пользователям просматривать ревизии совместно используемого объекта. Конфликтующие ревизии согласуются и объединяются в текущую версию совместно используемого объекта. Клиент может запрашивать текущую версию совместно используемого объекта из хранилища, когда текущая версия совместно используемого объекта недоступна из хранилища. Текущая версия совместно используемого объекта синхронизируется с клиентом, когда хранилище принимает текущую версию совместно используемого объекта. 3 н. и 7 з.п. ф-лы, 10 ил.

СПОСОБ И СИСТЕМА ДЛЯ СИНХРОНИЗАЦИИ МНОЖЕСТВЕННЫХ ПОЛЬЗОВАТЕЛЬСКИХ РЕВИЗИЙ СОВМЕСТНО ИСПОЛЬЗУЕМОГО ОБЪЕКТА

... 1. Компьютерно-реализуемый способ синхронизации множественных пользовательских ревизий совместно используемого объекта, содержащий этапы, на которых принимают ревизию совместно используемого объекта, определяют, конфликтует ли ревизия с синхронизированной ревизией на исходной странице совместно используемого объекта, и синхронизируют ревизию с совместно используемым объектом, когда определено, что ревизия связана с текущей версией совместно используемого объекта, и когда определено, что ревизия не конфликтует с синхронизированной ревизий. 2. Компьютерно-реализуемый способ по п.1, дополнительно содержащий этапы, на которых отображают ревизию на конфликтной странице, связанной с исходной страницей, когда определено, что ревизия конфликтует с синхронизированной ревизией, отображают индикатор конфликта на исходной странице, причем индикатор конфликта указывает, что исходная страница связана с конфликтной страницей, и включают ревизию в совместно используемый объект. 3. Компьютерно-реализуемый ...

ВЕДЕНИЕ ЖУРНАЛА ОТКАТА ДЛЯ СЕКЦИОНИРОВАННЫХ НАБОРОВ ДАННЫХ В ПАМЯТИ

Изобретение относится к средствам хранения данных. Технический результат заключается в сокращении времени обработки запросов в случае сбоя работы устройства. Устройство (10) хранения данных содержит модуль (100) управления, первый модуль (110) хранения, второй модуль (120) хранения, третий модуль (125) хранения и буфер (130). Первый модуль (110) хранения выполнен с возможностью хранения множества наборов данных. Модуль (100) управления выполнен с возможностью приема запросов на операции, подлежащие выполнению на множестве наборов данных в первом модуле (110) хранения, исполнения принятых запросов и копирования множества наборов данных в первом модуле (120) хранения во второй модуль (120) хранения в заданный момент времени. Модуль управления дополнительно выполнен с возможностью добавления принятых запросов на операции в буфер (130) и сохранения буфера (130) в третьем модуле (125) хранения с помощью средства синхронизации. Устройство хранения данных выполнено с возможностью инициализации ...

СПОСОБ УПРАВЛЕНИЯ СИСТЕМОЙ БАЗ ДАННЫХ (ВАРИАНТЫ)

... 1. Способ управления системой баз данных, в которую входит один или несколько узлов, содержащих один или несколько процессоров и память, в которой хранится одна или несколько программ, выполняемых одним или несколькими процессорами, включающий:прием энного числа запросов от одного или нескольких узлов системы баз данных;объединение энного числа запросов до инициирования операций обслуживания энного числа запросов;сокращение числа операций обслуживания энного числа запросов ивыполнение сокращенного числа операций обслуживания энного числа запросов.2. Способ по п.1, в котором система баз данных содержит поисковый механизм.3. Способ по п.2, в котором поисковый механизм содержит децентрализованную роевую систему, в которой используется один или несколько роевых алгоритмов.4. Способ по п.1, в котором:энное число запросов включает энное число запросов суммирования для суммирования со значением в строке, хранящейся в системе баз данных;при объединении энного числа запросов до инициирования операций ...

МАСШТАБИРУЕМАЯ КЛАСТЕРНАЯ БАЗА ДАННЫХ

... 1. Способ управления системой баз данных, в которую входит один или несколько узлов, содержащих один или несколько процессоров и память, в которой хранится одна или несколько программ, выполняемых одним или несколькими процессорами, включающий:прием энного числа запросов от одного или нескольких узлов системы баз данных;объединение энного числа запросов до инициирования операций обслуживания энного числа запросов;сокращение числа операций обслуживания энного числа запросов; ивыполнение сокращенного числа операций обслуживания энного числа запросов.2. Способ по п.1, в котором система баз данных содержит поисковый механизм.3. Способ по п.2, в котором поисковый механизм содержит децентрализованную роевую систему, в которой используется один или несколько роевых алгоритмов.4. Способ по п.1, в котором:энное число запросов включает энное число запросов суммирования для суммирования со значением в строке, хранящейся в системе баз данных;при объединении энного числа запросов до инициирования операций ...

СПОСОБЫ И УСТРОЙСТВО ДЛЯ РАСПРЕДЕЛЕННОЙ БАЗЫ ДАННЫХ, КОТОРАЯ ПОЗВОЛЯЕТ УДАЛЯТЬ СОБЫТИЯ

ASYNCHRONES LOKALES UND ENTFERNT ANGEORDNETES ERZEUGEN VON KONSISTENTEN ZEITPUNKT-MOMENTKOPIEN IN KONSISTENZGRUPPEN

Eine primäre Speichersteuereinrichtung zur Verwendung mit einem Host und einem primären Speichersystem an einem ersten Ort, wobei das primäre Speichersystem eine primäre Speichersteuereinrichtung und zumindest eine Speichereinheit aufweist, die durch die primäre Speichersteuereinrichtung gesteuert wird und dazu ausgebildet ist, eine Mehrzahl von Dateneinheiten zu speichern, und mit einem sekundären Speichersystem an einem zweiten, von dem ersten Ort entfernt angeordneten Ort, wobei das sekundäre Speichersystem eine sekundäre Speichersteuereinrichtung und zumindest eine Speichereinheit aufweist, die durch die sekundäre Speichersteuereinrichtung gesteuert wird und dazu ausgebildet ist, eine Mehrzahl von Dateneinheiten zu speichern, wobei die primäre Speichersteuereinrichtung eine Datenreplikationslogik aufweist, die eine Datenspiegelungslogik, eine Konsistenzgruppenlogik, eine Momentkopie-Erzeugungslogik, eine Zustandsermittlungslogik und eine Verzögerungslogik beinhaltet.

Benutzergesteuerte Replikation in einem System für synchronisierte Objektreplikationen

Eine Momentaufnahme ausgewählter Objekte in einem Quellen-Repository wird als Reaktion auf die benutzergesteuerte Replikation erstellt. Die Momentaufnahme wird zu einem Momentaufnahme-Replikationsjob bestimmt. Der Momentaufnahme-Replikationsjob wird zum Ende einer Replikationswarteschlange hinzugefügt, um die Ausführung für die synchronisierte Objektreplikation zu erwarten. Unsynchronisierte Objekte in einem Bestimmungsziel werden durch Vergleichen eines Zustands der ausgewählten Objekte in der Momentaufnahme mit einem aktuellen Zustand des Bestimmungsziels zur Zeit der Ausführung des Momentaufnahme-Replikationsjobs erkannt. Die unsynchronisierten Objekte im Bestimmungsziel werden auf der Grundlage des Vergleichs des Zustands der ausgewählten Objekte in der Momentaufnahme mit dem aktuellen Zustand des Bestimmungsziels zur Zeit der Ausführung des Momentaufnahme-Replikationsjobs synchronisiert.

Medical image data e.g. computed tomograpy data, storing and providing method for hospital, involves replicating marked partial metadata from central database to local database in decentralized manner to provide image data to satellites

The method involves acquiring image data (BD) at modality, where the image data comprise meta data e.g. patient identification number, and pixel data. Partial metadata in the metadata are marked. The metadata acquired by satellites (S1-S4) are locally stored in a local database (lokDB). The metadata stored in the local database are completely and centrally replicated in a central database (zentDB). The marked partial metadata are partially and automatically replicated from the central database to the local database in a decentralized manner to provide the image data to the satellites. Independent claims are also included for the following: (1) a computer program product comprising instructions for performing a method for storing and providing medical image data (2) a system for storing medical image data in a computer-based clinical facility.

Throttling mechanism

A high-throughput algorithm for multiversion concurrency control with globally synchronized time

A system and method for preserving throughput in a distributed system while maintaining concurrency. The system comprising a server device, the server device including at least one input/output to communicate with other servers and in a distributed computing environment, the server device comprising one or more processors configured to receive a request to write first data, write the first data to a memory in the distributed computing environment and while the first data is being committed to the memory, the effects of the first data are not visible to at least one of a client library, the other servers, or the clients for an interval of time, without preventing writing of data to the memory during the interval of time. The effects of the data may no be visible to both the client library and the other server during the interval of time. The client library may notify one of the clients after the interval of time has passed and the first data has been committed.

Sync field filtering

Rollback support in distributed data management systems

A high-throughput algorithm for multiversion concurrency control with globally synchronized time

Global table management operations for multi-region replicated tables

Systems and methods are disclosed to implement, in a database system, global table management operations for global tables or objects that are replicated across multiple locations. In embodiments, for a regional replica of a global table, a requested change to a metadata property of the replica (e.g., an attribute data type, a write throughput limit, or a time-to-live setting) is propagated to other replicas of the global table at other locations. In embodiments, before performing the requested management operation, the database system may first verify that the operation can be successfully performed in all locations. In embodiments, the operation may be performed optimistically without verification, and rolled back if it cannot be performed everywhere. In embodiments, to serialize management operations on a global table, the database system may check for other pending management operations via a shared control object, and proceed when any pending operations are finished.

Batch job performance immprovement in active-active architecture

In an approach for improving performance of a batch job (232) running on database servers in an active-active architecture. A processor (504), responsive to the batch job (232) being ready to be executed on a source database server (210), sends a first communication to a target database server (220) with a synchronization start point (410). While executing the batch job (232), the processor (504) utilizes a pre-lock function, pre-load function, and lock avoidance function to prevent lock conflicts (420). The processor (504), responsive to either the source database server (210) or the target database server (220) encountering a commit statement, suspends the respective database server and sends a second communication to ask if the other respective database server is ready to complete the commit statement (430). The processor (504), responsive to the other respective database server confirming its ready to complete the commit statement, completes the commit statement on both the source database ...

PROCEDURE AND SYSTEM FOR THE ACTUALIZATION OF GEOGRAPHICAL DATA BASES

PROCEDURE FOR THE DATA ADMINISTRATION IN A NETWORK PARTIAL OF MADE A REPLICATION DATA BASE SYSTEMS

Asynchronous distributed object uploading for replicated content addressable storage clusters

A method is performed by two or more devices of a group of devices in a distributed data replication system. The method includes receiving, at the two or more devices, a group of chunks having a same unique temporary identifier, where the group of chunks comprises an object to be uploaded; creating an entry for the object in a replicated index, where the entry is keyed by the unique temporary identifier, and where the replicated index is replicated at each of the two or more devices; and determining, by an initiating device of the two or more devices, that a union of the group of chunks contains all data of the object. The method also includes calculating a content-based identifier to the object; creating another entry for the object in the replicated index, where the other entry is keyed by the content-based identifier; and updating the replicated index to point from the unique temporary identifier to the content-based identifier.

System and method for synchronizing multiple calendars over wide area network

ASYNCHRONOUSLY PROCESSING SEQUENTIAL DATA BLOCKS

A data analytics system stores a plurality of sequential data blocks in a block cache. Each data block contains one or more data records. Each data record of a data record contains one or more values and a record marker marking the end of the data record. The data analytics system allocates a data block in the block cache to a worker thread. The worker thread parses the data block to identify the one or more data records contained within the data block. The data analytics system determines whether the data block was correctly parsed. Responsive to determining that the data block was correctly parsed, the data analytics system provides the one or more data records contained within the data block to a data analytics workflow.

METHODS AND APPARATUS FOR A DISTRIBUTED DATABASE THAT ENABLES DELETION OF EVENTS

In some embodiments, an apparatus includes a memory associated with an instance of a distributed database at a compute device configured to be included within a first group of compute devices. The apparatus is configured to determine an order for each event from the set of events based on different configurations of an event consensus protocol. The different configurations are logically related to different configurations of compute devices that implement the distributed database. The apparatus is configured to determine a current state of the instance of the distributed database based on the order determined for each event from the set of events and generate a signed state associated with the instance of the distributed database based on a hash value associated with the current state. The apparatus sends a signal to post into the instance of the distributed database an event that includes a transaction indicative of the signed state.

INFORMATION MANAGEMENT UPDATING SYSTEM

A system and method for updating information of the user comprising a first database containing information representing a plurality of users and a user interface operable to receive an updated information of the user and a network interface adapted to access a network. The system further comprises a processing circuit operably coupled to the first database, the user interface and the network interface wherein the processing circuit is adapted to transmit to a second database through the network at least one updated record to match a corresponding record in the first database.

SYSTEM AND METHOD FOR CREATING, DEPLOYING, INTEGRATING, AND DISTRIBUTING NODES IN A GRID OF DISTRIBUTED GRAPH DATABASES

In a computer environment, a system and method is described that generally provides for creation of a distributed graph database, creation and deployment of nodes in a distributed graph database system, and integration of nodes into a set of distributed graph databases that include data nodes and edges that are: entities built using forms, relations, and relationships; immutable but evolvable through the addition of new data nodes or new edges joining the evolving data node to another data node; shareable and mergeable.

APPARATUS, AND ASSOCIATED METHOD, FOR SYNCHRONIZING A DATABASE

An apparatus, and an associated method, facilitates synchronization of a database formed of data records. When a synchronization response to a synchronization request indicates that the data of the request is unsynchronizable, data associated with the synchronization request is marked. A retry procedure is subsequently utilized to attempt synchronization of the marked data records and to determine the individual data records that cannot be synchronized.

SYSTEMS AND METHODS FOR DISTRIBUTED STORAGE

Techniques for distributed storage using a plurality of computing devices connected to a network can include storing an electronic file in a local storage layer of one of the computing devices. The stored electronic file can be asymmetrically transmitted, in portions, over the network to other computing devices to store the file across the other computing devices in a distributed storage layer. The electronic file can be asynchronously transmitted over the network to a cloud storage layer such that the electronic file is mirrored in the cloud storage layer. The local storage layer of each computing device can store, for each electronic file stored in the distributed storage layer, metadata having pointers to locations of the portions the electronic files stored in the local storage layer and distributed storage layer. The electronic files stored in the distributed storage layer can be displayed as stored in a single logical drive.

APPARATUS AND METHOD FOR SHARING A DATA OF MOBILE TERMINAL

system, distributed data replication methods is carried out by one or more devices of a plurality of devices and computer readable memory.

Extracting incremental data

The present disclosure introduces a method, an apparatus, and a system for extracting incremental data. Primary key information of incremental data is obtained from a backup database. The incremental data is inquired based on the primary key information from a main database that synchronizes with the backup database. The found incremental data is then inserted into a target data warehouse. The present techniques not only save a lot of time and system resources but also improve the efficiency of incremental data extraction.

HANDLING OF DATA ARCHIVING EVENTS IN A REPLICATION SYSTEM

Systems and methods include detection of deletion of a record of a database table due to a database transaction, creation of a log entry including one or more key field values of the record of the database table, an identifier of a user associated with the database transaction, and an indicator of a delete operation, determination of whether the identifier identifies an archiving user, and, in a case that the identifier identifies an archiving user, change the indicator to an indicator of an archive operation to create a changed log entry.

Method and apparatus for folder synchronization and management

A method of sharing data accessible by a server, with a portable electronic device of a first entity includes receiving permission at the server to share with the first entity, data associated with a second entity, the data for use with an application. The server retrieves the data associated with the second entity and transmits the data from the server to the portable electronic device of the first entity. The data associated with the second entity is received at the portable electronic device of the first entity and stored in association with a second entity identifier, thereby distinguishing the data from other data for use with the application.

Pending state management for mobile business objects

A system, method, and computer program product are provided for invoking an operation to change a current data row of a table in a local database. As a result, a pending change data row is inserted in the table, distinguished by a primary key comprising a common key value and a pending change indicator. The pending change is then submitted for synchronization.

Synchronization of database changes among multiple devices

Methods, systems, and apparatus, including computer program products, for synchronizing database changes among multiple devices. In one aspect, a method includes determining one or more primary differences between a primary database and a local synchronization database; receiving, from a secondary device, one or more secondary differences associated with a secondary database; merging the primary differences and the secondary differences to generate a set of merged differences; providing the set of merged differences to the secondary device; receiving, from the secondary device, an acknowledgment indicating receipt of the set of merged differences; and updating the primary database in accordance with the set of merged differences after receipt of the acknowledgment. Further, at least one conflict can be detected between the primary differences and the secondary differences, and can be resolved prior to generating the set of merged differences.

DATA PROTECTION METHODS AND SYSTEMS FOR A NETWORKED STORAGE ENVIRONMENT

Data protection methods and systems for a storage environment are provided. A first-in-first out (FIFO) structure stores a logical representation of a first storage location that retains previous data for a data container, after new data for the data container is stored at a second storage location. The FIFO structure also stores a logical representation of a file system tree structure that is stored in persistent storage, after a consistent point operation. In response to an event, the file system tree structure is selected, based on the file system tree structure being closest to a transaction. A snapshot is generated using the file system tree structure. Thereafter, the data container is restored from the snapshot or from a copy of the snapshot. 1. A method , comprising:retaining, by a processor, in a first-in-first out (FIFO) structure, a logical representation of a first storage location that stores previous data for a data container, prior to writing new data for the data container at a second storage location;utilizing, by the processor, the FIFO structure to point to a previous version of a file system tree structure stored at persistent storage, before the previous data is written at the first storage location;selecting, by the processor, the previous version of the file system tree structure to restore the data container to a transaction, based on a proximity of the previous version of the file system tree structure to the transaction;executing, by the processor, a snapshot operation for taking a snapshot of the previous version of the file system tree structure; andutilizing, by the processor, the snapshot to restore the data container.2. The method of claim 1 , wherein the data container is a database.3. The method of claim 1 , further comprising: updating claim 1 , by the processor claim 1 , an active file system tree structure upon writing the new data.4. The method of claim 3 , further comprising: selecting claim 3 , by the processor claim 3 , the ...

Simplified application object data synchronization for optimized data storage

A method for synchronizing a mobile device with a central computer system is provided. The method includes coupling the mobile device to an application server of the central computer system over a network, defining a sync list that includes a candidate object type for distribution, synchronizing a relevant object of the candidate object type to the mobile device, determining a referenced object type from the relevant object type, and synchronizing a referenced object of the referenced object type to the mobile device. The sync list can be defined from meta-data that associates a user of the mobile device with one or more candidate object types, where the candidate object types serve as a starting point for the distribution process. Referenced object types can be determined from the relevant object types through additional meta-data.

SYSTEMS AND METHODS FOR AUTO DISCOVERY OF FILTERS AND PROCESSING ELECTRONIC ACTIVITIES USING THE SAME

The present disclosure relates to systems and methods for filtering electronic activities. Exemplary implementations may include ingesting a first electronic activity; identifying an associated entity; and selecting a first filtering model based on the entity, the first filtering model trained to indicate whether to restrict further processing of ingested electronic activities. The method may further include generating a plurality of structured data tags for the first electronic activity; applying the selected first filtering model to the plurality of structured data tags for the first electronic activity to determine whether the first electronic activity satisfies a first restriction condition; and responsive to the first electronic activity satisfying the first restriction condition, restricting the first electronic activity from further processing; or responsive to the first electronic activity not satisfying the first restriction condition, further processing, by the one or more processors ...

Method and apparatus for maintaining replica sets

Provided are systems and methods for managing asynchronous replication in a distributed database environment, wherein a cluster of nodes are assigned roles for processing database requests. In one embodiment, the system provides a node with a primary role to process write operations against its database, generate an operation log reflecting the processed operations, and permit asynchronous replication of the operations to at least one secondary node. In another embodiment, the primary node is the only node configured to accept write operations. Both primary and secondary nodes can process read operations. Although in some settings read requests can be restricted to secondary nodes or the primary node. In one embodiment, the systems and methods provide for automatic failover of the primary node role, can include a consensus election protocol for identifying the next primary node. Further, the systems and methods can be configured to automatically reintegrate a failed primary node.

TABLE PARTITIONING WITHIN DISTRIBUTED DATABASE SYSTEMS

Techniques are disclosed for implementing a unified partitioning scheme within distributed database systems to allow a table to be horizontally partitioned and those partitions stored on and serviced by a storage group. A storage group is a subset of storage manager (SM) nodes, and each SM node is configured to persist database data in durable storage. The distributed database system assigns each storage group to a subset of SM nodes. The distributed database system can address each storage group using a symbolic mapping that allows transactions to identify a particular storage group, and to direct read and write operations to a subset of SM nodes servicing that storage group. An administrator can update this mapping on-the-fly to cause the distributed database system to dynamically adjust an implemented partitioning scheme without necessarily interrupting on-going database operations.

MOBILE APPLICATION MIGRATION TO CLOUD COMPUTING PLATFORM

Technology is disclosed herein for migrating at least portions of computer applications to a server. According to at least one embodiment, a computing device identifies a service component of a computer application executed at the computing device. The service component does not directly interact with a user of the computing device. The computing device transfers an instance of the service component to a server so that the service component can be executed at the server. The computing device then receives from the server a result message generated by the instance of the service component executed at the server. The computing device feeds the result message to a user interface component of the computer application via a procedure call such that the user interface component retrieves the result message as if it were generated by an instance of the service component executed at the computing device.

Role assignments in a cloud infrastructure

Provisioning, managing and tracking of services provided by a cloud infrastructure system are described. A subscription order request from a customer for services provided by the cloud infrastructure system is received. A separation of roles performed by different customer entities in a customer's organization when the customer subscribes to services in the cloud infrastructure system is determined. The roles relate to providing information about different portions of the subscription order by the customer entities. Upon completion of the assigned roles by the relevant customer entities, the subscription order is provisioned to the customer.

Methods and Systems for Deleting Large Amounts of Data From a Multitenant Database

A bulk delete for remote database access is provided. A delete request is received over an API for a multitenant database. The delete can be for either a soft delete or a hard delete, where hard delete allows the system to skip use of a recycle bin. Multiple batch jobs are created to implement the delete request, and the batch jobs are executed asynchronously with respect to the request until the request is satisfied. Results for each batch job can be stored and provided to a client to indicate results of the delete operation.

Method and system for server synchronization with a computing device

A method and system for synchronizing a main database of a server and a local database of a handheld device. A user can use an application residing in the handheld device to make transactions in the local database. During a synchronization operation, the handheld device and server are coupled. The system then determines whether the application should be updated and, if so, causes the server to provide an update. The system also causes the handheld device to provide to the server information related to the transactions made by the user to the local database. The system causes the server to perform transactions on the main database based on the transaction information. The system also causes the server to extract data from the main database. The server can then provide at least some of the extracted data to the handheld device to update the local database.

PROVIDING CONSISTENCY IN A DISTRIBUTED DATA STORE

A request to store a new value to a data entity to be fault tolerant across a plurality of nodes is received. The data entity being in an intermediate state is indicating to at least one less than a majority of the plurality of nodes. A request to read a stored value of the data entity is received. A latest value instance of the data entity among a majority nodes of the plurality of nodes is determined. A value of the latest value instance in a majority nodes of the plurality of nodes is stored as a next value instance of the data entity.

Method and framework for transaction synchronization

A method and framework for transaction synchronization. An embodiment of the present invention first involves "cloning" the middle tier business logic layer from a server onto a client. The framework allows the cloned business logic on the client and the business logic layer on the server to operate independently without a connection. Therefore, the server-side and client-side applications can fully operate without a connection. When a connection is established, the framework synchronizes (reconciles according to defined rules) transactions (i.e., execution of functions) so that transactions resulting from the execution of a function on the business logic layer on the client is equivalent to a transaction resulting from a execution of a function on the business logic layer on the server. Embodiments of the invention may also be used in peer-to-peer environments.

METHOD, DEVICE, AND APPARATUS FOR WORD VECTOR PROCESSING BASED ON CLUSTERS

A cluster includes a server cluster and a worker computer cluster. Each worker computer included in the worker computer cluster separately obtains a word and at least one context word of the word that are extracted from a corpus. The worker computer obtains word vectors for the word and the at least one context word. The worker computer calculates a gradient according to the word, the at least one context word, and the word vectors. The worker computer asynchronously updates the gradient to a server included in the server cluster. The server updates the word vectors for the word and the at least one context word of the word according to the gradient.

Data synchronization via three or more paths

A server is used to send data records to clients. A client #1 may synchronize with the server using a first path and a first protocol. The client #2 may synchronize with the server using a second path and a second protocol. In addition to synchronizing with the server, client #1 and client #2 may synchronize data records with each other using a third path and a third protocol. When synchronizing through different paths and/or protocols, data record identifiers may be generated differently. Thus, devices may not be able to synchronize data records if the data records are identified differently by client #1, client #2, and/or the server. A universal identity is used to allow for synchronization through three paths and protocols. Thus, client #1 and client #2 may synchronize data records through the third path and third protocol using the universal identity for data records.

SYSTEMS AND METHODS FOR AUTO DISCOVERY OF FILTERS AND PROCESSING ELECTRONIC ACTIVITIES USING THE SAME

The present disclosure relates to systems and methods for filtering electronic activities. Exemplary implementations may include ingesting a first electronic activity; identifying an associated entity; and selecting a first filtering model based on the entity, the first filtering model trained to indicate whether to restrict further processing of ingested electronic activities. The method may further include generating a plurality of structured data tags for the first electronic activity; applying the selected first filtering model to the plurality of structured data tags for the first electronic activity to determine whether the first electronic activity satisfies a first restriction condition; and responsive to the first electronic activity satisfying the first restriction condition, restricting the first electronic activity from further processing; or responsive to the first electronic activity not satisfying the first restriction condition, further processing, by the one or more processors ...

SYSTEM AND METHOD FOR CREATING, DEPLOYING, INTEGRATING, AND DISTRIBUTING NODES IN A GRID OF DISTRIBUTED GRAPH DATABASES

In a computer environment, a system and method is described that generally provides for creation of a distributed graph database, creation and deployment of nodes in a distributed graph database system, and integration of nodes into a set of distributed graph databases that include data nodes and edges that are: entities built using forms, relations, and relationships; immutable but evolvable through the addition of new data nodes or new edges joining the evolving data node to another data node; shareable and mergeable.

Systems and methods for optimizing performance of graph operations

A method of optimizing graph operations is performed by a computing system. The method comprises: (1) receiving a first request to perform a first operation on a first graph, where the first graph comprises a set of vertices and a set of edges, each edge connecting a pair of vertices, and each vertex having one or more associated properties; (2) logging the first request, but not performing the first operation; (3) receiving a second request to perform a second operation; (4) logging the second request, but not performing the second operation; (5) receiving a query for data from the first graph, where the data includes property values for one or more vertices; (6) in response to the query: (a) generating a second graph by optimizing and performing the first and second operations; and (b) returning data responsive to the query, where the returned data is based on the second graph.

Systems and methods for distributed storage

Techniques for distributed storage using a plurality of computing devices connected to a network can include storing an electronic file in a local storage layer of one of the computing devices. The stored electronic file can be asymmetrically transmitted, in portions, over the network to other computing devices to store the file across the other computing devices in a distributed storage layer. The electronic file can be asynchronously transmitted over the network to a cloud storage layer such that the electronic file is mirrored in the cloud storage layer. The local storage layer of each computing device can store, for each electronic file stored in the distributed storage layer, metadata having pointers to locations of the portions the electronic files stored in the local storage layer and distributed storage layer. The electronic files stored in the distributed storage layer can be displayed as stored in a single logical drive.

Asynchronous distributed garbage collection for replicated storage clusters

A method may be performed by a device of a group of devices in a distributed data replication system. The method may include storing objects in a data store, at least one or more of the objects being replicated with the distributed data replication system, and conducting a scan of the objects in the data store. The method may further include identifying one of the objects as not having a reference pointing to the object, storing a delete negotiation message as metadata associated with the one of the objects, and replicating the metadata with the delete negotiation message to one or more other devices of the group of devices.

Method and system for sandbox visibility

A method for local data visibility is disclosed. The method includes detecting data items in a local data repository located in a local environment. In response to the detecting, a computer processor of the local environment collects metadata describing the data items in the local data repository. The metadata includes a project identifier. The metadata is then transmitted, separate from the data items, to a data manager of a database, where the database is accessible by multiple local environments. Accordingly, the data manager determines, based on the project identifier, a synchronization status of the data items in the local data repository, where the synchronization status represents a relationship between the data items in the local data repository and exploration and production data in the database. The data manager further generates, according to a pre-determined data management scheme, an alert based on the synchronization status.

Continuous data replication

In a first embodiment, a method and computer program product for use in a storage system comprising quiescing IO commands the sites of an ACTIVE/ACTIVE storage system, the active/active storage system having at least two storage sites communicatively coupled via a virtualization layer, creating a change set, unquiescing IO commands by the virtualization layers, transferring data of a change set to the other sites of the active/active storage system by the virtualization layer, and flushing the data by the virtualization layer. In a second embodiment, a method and computer program product for use in a storage system comprising fracturing a cluster of an active/active storage system; wherein the cluster includes at least two sites, stopping IO on a first site of the cluster; and rolling to a point in time on the first site.

Method for Sharing and Searching Playlists

A method for making a playlist available to the public, in which the playlist comprises user-defined descriptor information. The user-defined descriptor information is entered as free form text or prose.

Parallel database mirroring

In a database cluster comprising a plurality of mirrored database segments, a mirror database pair comprising a primary database and a mirror database are maintained substantially consistent by writing changes resulting from transactions to page data in parallel to the primary database and to the mirror database. Changes are sent to the mirror database over a network using network communications. Transaction logs are also written to the primary and mirrored databases in parallel, the transaction logs being sent to the mirror database using network communications. Bulk load changes resulting from transactions are written in parallel over the network to the primary database and the mirror database without writing the changes to a transaction log. In the event of a failure, resynchronization of the mirror database can be performed quickly while avoiding a lengthy ongoing redo process on the mirror database.

SYSTEMS AND METHODS FOR GENERATING A FILTERED DATA SET

The present disclosure relates to generating a filtered data set. Data from a plurality of systems of record of a plurality of data source providers may be accessed. A master data set generated using the data accessed from the plurality of systems of record may be maintained. Restriction policies including one or more rules for restricting sharing of data may be maintained. A filtered data set may be generated for a data source provider responsive to an application of restriction policies of other data source providers to the master data set. The filtered data set may be provisioned.

SYSTEMS AND METHODS FOR SYNCHRONIZATION EVENT BUILDING AND/OR COLLAPSING BY A SYNCHRONIZATION COMPONENT OF A CLOUD-BASED PLATFORM

Techniques for monitoring local and/or remote file systems by a synchronization component (e.g., client/server) of a cloud-based platform are disclosed. In some embodiments, a method of building synchronization events by a synchronization component (e.g., a synchronization server/client) includes obtaining a set of items that have been changed and their new states and retrieving last known states of the set of items that are stored in a reference snapshot inside a filesystem scanner. The method further includes generating differences between the new states and the last known states of the set of items as item changes and utilizing information provided by the item changes to translate the item changes into synchronization events for execution on the opposing file system. A method of handling failed synchronization events by a synchronization component of the cloud-based platform by collapsing a subsequent event with the failed synchronization event is also disclosed.

Executing transactions on distributed databases

A distributed database system executes transactions on a distributed database. A received transaction includes statements describing modifications of records stored in the distributed database. The distributed database system executes the transaction at a query server by obtaining copies of records corresponding to the statements of the transaction and performing the modifications specified by the statements of the transaction on the record copies. The distributed database system stores the modified record copies at the query server during execution of the transaction. After the transaction has successfully been executed at the query server, the distributed database system attempts to perform a commit process to update the records stored in the distributed database based on the modified record copies.

Method and system for attributing metrics in a CRM system

In an embodiment of the present invention, a Response object collects and synchronizes information from other types of objects as may be implemented in a CRM system. For example, certain objects may include a Lead or Contact Object, a Campaign object, and an Opportunity Object that is synchronized into a Response object. In this way, the results for a given Lead or Contact can be directly attributed to the Campaign or Opportunity without having to guess as may be necessary in certain typical CRM systems.

ОБНОВЛЕНИЕ И РЕПЛИКАЦИЯ РЕСУРСОВ

Изобретение относится к области обновления и репликации ресурсов. Техническим результатом является повышение гибкости системы репликации ресурсов за счет обеспечения возможности различных режимов репликации. Способ выявляет, когда локальные ресурсы устаревают. Локальные ресурсы могут быть устаревшими, когда время между последним успешным действием синхронизации и текущим временем превышает некоторое значение устаревания. Если локальные ресурсы определены как устаревшие, то локальный участник может воздерживаться от некоторых действий синхронизации до тех пор, пока этот локальный участник не получит возможность возобновлять эти некоторые действия синхронизации. Если локальные ресурсы определены как неустаревшие, то могут выполняться дополнительные действия синхронизации, включая синхронизацию ресурсов между данным участником и партнером-источником или целевым партнером. 3 н. и 11 з.п. ф-лы, 6 ил.

СПОСОБ И УСТРОЙСТВО ДЛЯ СИНХРОНИЗАЦИИ ДАННЫХ

Изобретение относится к технологиям сетевой связи. Технический результат заключается в повышении скорости передачи данных. Способ содержит прием облачным сервером запроса на синхронизацию данных, переданного подвижным терминалом, причем запрос на синхронизацию содержит идентификатор подвижного терминала, время синхронизации данных подвижным терминалом в прошлый раз и учетную запись синхронизации для синхронизации данных между подвижным терминалом и облачным сервером; определение облачным сервером, что время операции записи подлежащих синхронизации данных больше чем время синхронизации данных подвижным терминалом в прошлый раз, при этом подлежащие синхронизации данные являются данными, которые соответствуют идентификатору в учетной записи синхронизации и находятся на облачном сервере; и синхронизацию облачным сервером подлежащих синхронизации данных с подвижным терминалом. 2 н. и 10 з.п. ф-лы, 9 ил.

СЕРИАЛИЗАЦИЯ КОМАНД РЕДАКТИРОВАНИЯ ДОКУМЕНТА

Изобретение относится к средствам сериализации команд редактирования документа. Технический результат заключается в уменьшении появления ошибок при согласованности изменений между двумя версиями документами. Сохраняют документ на компьютере-сервере. Принимают множество пользовательских команд, используя основывающиеся на web приложения, предписывающих модифицировать документ. Выполняют сериализацию принятых команд. Сохраняют сериализованные команды в последовательном порядке в потоке команд отдельно от документа. Определяют, является ли компьютер-сервер сильно загруженным компьютером-сервером. Если компьютер-сервер является сильно загруженным компьютером-сервером, то определяют, есть ли выполняющаяся в текущий момент сессия редактирования документа для данного документа. Если есть выполняющаяся в текущий момент сессия редактирования документа, применяют поток команд к документу. Переносят документ на несильно загруженный компьютер-сервер. 3 н. и 14 з.п. ф-лы, 9 ил.

SPEICHERUNGS-VERWALTUNGSSYSTEM ZUM ERHALTEN DER EINHEITLICHKEIT VON FERNKOPIEDATEN

Triangular asynchronous replication

A high-throughput algorithm for multiversion concurrency control with globally synchronized time

A method of maintain concurrency in a distributed computer system by pushing a commit wait period to client commit paths and to future readers. As opposed to servers performing commit waits, the servers may assign timestamps, which are used to ensure that causality is preserved. When a server executes a transaction that writes data to a distributed database, the server acquires a user-level lock, and assigns the transaction a timestamp equal to a current time plus an interval corresponding to bounds of uncertainty of clocks in the distributed system. After assigning the timestamp, the server releases the user-level lock. Any client devices, before performing a read of the written data, must wait until the assigned timestamp is in the past.

Live migration of applications and file systems in a distributed system

When migrating applications from one server to another, a protocol handler pauses incoming requests during the migration. Outstanding requests may be terminated after a timeout period. Alternatively, the paused requests may be released if the outstanding requests have not completed after a timeout period. When replicating a file system, snapshots of a source file system are taken at predetermined points in time. These are replicated to the destination file system. After a period when the file system could not be replicated, the most recent version of the file system is identified. The last common snapshot is also identified. Subsequent snapshots are then replicated from the most recent version to the older version. The most recent version may be identified from the average age of the snapshots and the number of changes in the snapshots. Only the snapshots unique to a particular server may be used to determine the most recent version.

Data security device

Transfer mode control program, transfer mode control device, and transfer mode control method

Monotonic transactions in multi-master database with loosely coupled nodes

Tracking change data capture log history

A method of tracking change data capture (CDC) log history comprises obtaining a first snapshot of a source system and deriving a set S1 of key-value pairs reflecting the first snapshot. A mirror operation of the source system is performed to obtain CDC change operations representing changes with respect to the set S1, the CDC change operations captured as a set SM of key-value pairs. A first CDC log is obtained as a first sequence SA of key-value pairs including the key-value pairs of the set S1 and the set SM. A second snapshot of the source system is obtained and a set S2 of key-value pairs reflecting the second snapshot is derived. The first sequence SA is compared with the set S2 to derive corrective CDC operations as a set S3 of key-value pairs, the corrective CDC operations representing corrections with respect to the first sequence SA. A second CDC log is obtained as a second sequence SB of key-value pairs including the key-value pairs of the sequence SA and the set S3. The corrective ...

PROCEDURE AND SYSTEM FOR THE FUNCTION SPECIFIC ONE AND TIME-CONFIGURABLE REPLIKATION OF PROCESSING FUNCTIONS

DEVICE AND ALLOCATION PROCEDURE FOR THE BLOCKAGE AND IDENTIFICATION OF A CHANGE LIST, WHICH IS USED ACCORDINGLY BY EXPIRATIONS OF DATA BASE SYNCHRONIZATION IN A RADIO COMMUNICATION SYSTEM

SYSTEMS AND PROCEDURES FOR CONTINUOUS ONES PIMSYNCHRONISATION BETWEEN A HOST COMPUTER AND A IN OF THE HAND CLIENT MECHANISM HELD

METHOD AND COMPUTER SYSTEM FOR ASYNCHRONOUS MESSAGING

METHOD AND APPARATUS FOR DATA TRANSMISSION

INTELLIGENT STORAGE DEVICE CONTROLLER

Asynchronous distributed garbage collection for replicated storage clusters

A method may be performed by a device of a group of devices in a distributed data replication system. The method may include storing objects in a data store, at least one or more of the objects being replicated with the distributed data replication system, and conducting a scan of the objects in the data store. The method may further include identifying one of the objects as not having a reference pointing to the object, storing a delete negotiation message as metadata associated with the one of the objects, and replicating the metadata with the delete negotiation message to one or more other devices of the group of devices.

Serializing document editing commands

A command stream is generated that includes serialized commands for editing a document. The command stream can be applied to a modified document to generate a single document that contains modifications to the document made using both a Web application and a client application. The command stream can also be utilized to recreate the edited state of a document following the disconnection from a Web application for editing the document, to load balance a server computer hosting the Web application even while editing sessions are in progress, to perform an upgrade of a server hosting the Web application while editing sessions are in progress, and for other purposes.

System and Method for Incrementally Replicating Investigative Analysis Data

A method of incrementally replicating investigative analysis data is disclosed along with a system for performing the same. The method and system provide the ability to break a data replication job into multiple "replication chunks" which can be exported and imported separately. By doing so, the method and system can efficiently replicate large numbers of data object changes over an unreliable data network. -29- ...

Cross-ACL Multi-Master Replication

Techniques for cross-ACL multi-master replication are provided. The techniques allow a replication site in a multi-master replication system implementing an asynchronous replication protocol and an access control policy to appropriately apply received data change updates to data maintained at the site even where a data change update is missing information because of the implemented access control policy. -57- C) Of C) C~) -j L 0 _ 0 ILL LL U5 > C-) CC-) U)I LU ...

Information management of mobile device data

A method and system for providing information management of mobile device data. In some examples, the system provides a user interface to permit a user of an information management system to define information management policies for the mobile device, receives definitions of the information management policies from the provided interface, and sends data from the mobile device to the information management system in accordance with the information management policies. In some examples, the system sends information identifying the user and the mobile device to the information management system. In some examples, the system sends the information management policies defined from the interface to the information management system. -60- ...

RECONCILIATION OF INDIRECTLY EXECUTED EXCHANGES OF DATA USING PERMISSIONED DISTRIBUTED LEDGERS

The disclosed exemplary embodiments include computer-implemented systems, apparatuses, and processes that monitor and reconcile indirectly initiated exchanges of data between network-connected devices and computing systems using a permissioned distributed ledger. For example, an apparatus may obtain and transmit first transaction information characterizing a data exchange to a first computing system. The first computing system may submit a portion of the first transaction information to a second computing system through a programmatic interface inaccessible to the apparatus, and the second computing system may execute the data exchange in accordance with at least the portion of the first transaction information. The apparatus may also obtain at least one element of a distributed ledger that includes encrypted second transaction information characterizing the execution of the data exchange, decrypt the encrypted second transaction data and perform operations that reconcile the first transaction ...

TECHNOLOGIES FOR SCALING USER INTERFACE BACKEND CLUSTERS FOR DATABASE-BOUND APPLICATIONS

Technologies for scaling user interface (UI) backend clusters for database-bound applications include a plurality of UI application servers and a master database server. Each UI application server is configured to retrieve a session sequence ID associated with a present user session that corresponds to a sequence ID of a master database of the master database server during a most recent write operation of data to a master database during the present user session. Additionally, the UI application servers are configured to read, from a local read-only database, a local database sequence ID that corresponds to a sequence ID of the master database during a most recent asynchronous replication of the master database to the local read-only database. The UI application servers are further configured to read the data from the local read-only database in response to a determination that the session sequence ID is empty, or is less than or equal to the local database sequence ID. Additional embodiments ...

APPARATUS AND METHODS OF DATA SYNCHRONIZATION

Various embodiments include apparatus and methods to synchronize virtualized data, or subsets of virtualized data, across a plurality of data repositories. The synchronization may be conducted in a data virtualization platform separate from the plurality of physical data repositories without requiring direct access to the plurality of physical data repositories. Additional apparatus, systems, and methods are disclosed.

SYNCHRONIZATION OF HIERARCHICAL DATA

The invention relates to the synchronization of hierarchical data between a transmitter and a receiver by the use of hash codes.

ASYNCHRONOUS INFORMATION SHARING SYSTEM

Techniques are disclosed for sharing information in a wide variety of contexts. An information sharing system is described that allows both an explicit capture process and an implicit capture process to add information items to a staging area. Further, the information sharing system supports both implicit and explicit consumption of information items that are stored in said staging area. A rules engine is provided to allow users to create and register rules that customize the behavior of the capture processes, the consuming processes, and propagation processes that propagate information from the staging areas to designated destinations. Techniques are also described for achieving exactly-once handling of sequence of items, where the items are maintained in volatile memory. Techniques are also provided for recording DDL operations, and for asynchronously performing operations based on the previously-performed DDL operations.

METHOD AND SYSTEM FOR MANAGEMENT OF SYNCHRONIZATION OF A MOBILE ELECTRONIC DEVICE

The present technology provides a method, system and computer program product for managing the synchronization of a mobile electronic device (330), wherein management and adjustment of the synchronization process (140) can be provided at least in part on a session by session basis. The synchronization process (140) may be configurable via a user interface. Aspects of the user interface, such as prompts or degree of user configurability, may be adjusted depending on one or more factors, such as link effective speed or expected synchronization time.

COMMUNICATION SYSTEM WITH PIM ENTRY SYNCHRONIZATION AND RELATED METHODS

A communication system (30) may include a mobile wireless communications device (35a-35c) associated with an account including an ordered master set (80) of personal information manager (PIM) entries (81a-81c) for a type of PIM data. The mobile wireless communications device may be configured to store an ordered subset (85) corresponding to the ordered master set of PIM entries for the type of PIM data. The communication system may also include a synchronization server (32) configured to synchronize the mobile wireless communications device and the account by mapping an PIM entry (81d) from the ordered master set to fill the ordered subset responsive to deletion of a PIM entry (86c) from the ordered subset. For example, the PIM data can include notes, journal entries, address book contents, lists, tasks, memos, calendar content (e.g. birthdays, anniversaries, appointments and meetings), reminders, account information (e.g. email, instant message and social networking account information ...

SEQUENCE ENGINE

According to one aspect, embodiments of the invention provide a data replicator comprising a system exit configured to capture data received at a source, a configuration control system configured to receive captured data from the system exit and modify the captured data to include sequence information, a message broker configured to receive the modified data from the configuration control system and to convert the modified data to a format of a destination, and a sequence engine configured to receive the converted and modified data from the message broker in a first sequence, to re-sequence, based on the sequence information, the converted and modified data in the first sequence into a second sequence, where the second sequence corresponds to the chronological order in which the data was captured by the system exit, and to forward the converted and modified data to the destination database in the second sequence.

METHOD, COMPUTER PROGRAM PRODUCT, AND SYSTEM FOR DISTRIBUTING CHANGES MADE IN A DATA STORE TO REMOTE CLIENT COPIES OF THE DATA STORE

A method, computer program product, and system that allows changes made to an original database table found on a server computer to be reflected in client copies of the database table based on intermittent client requests for synchronization. A server makes periodic updates of table differences between current table (20) receiving database change events and reference table (28). Each client copy of a database table and update (created by the server has a sequential version number associated therewith). The server will compare the version number of a client copy of a database table with the most recent version number of the table on the server to determine which updates need be applied in order to make the client copy current. Next, the updates will be translated from a generic format into instructions that are specific to the type of database engine being run on the client. Finally, the instructions are transmitted to the client (along with the new version number) so that the client may ...

For data backup method and device

SYNCHRONIZED DATA MANAGEMENT SYSTEM AND METHOD

GEOGRAPHICALLY DISTRIBUTED DATABASE

METHOD, DEVICE, AND APPARATUS FOR WORD VECTOR PROCESSING BASED ON CLUSTERS

Audio systems and related devices and methods

Systems and methods for processing data in security systems using parallelism, stateless queries, data slicing, or asynchronous pull mechanisms

Systems and methods for processing data using parallelism, stateless queries, data slicing, and/or asynchronous pull are provided. Some methods can include each of a plurality of data pull devices of a central server retrieving data from a respective one of a plurality of regional server databases, wherein a number of the plurality of data pull devices equals a number of the plurality of regional server databases, and wherein the plurality of data pull devices function asynchronously. Additionally or alternatively, some methods can include each of a plurality of threads of a central server retrieving a predetermined size of data from a plurality of regional server databases, wherein the plurality of threads are stateless so that any of the plurality of threads can retrieve the data from any of the plurality of regional server databases.

Method for replacing a currently operating data replication engine with a new data replication engine without application downtime and while preserving target database consistency, and by using audit trail tokens

An automated method is provided for replacing a currently operating data replication engine with a new data replication engine. The currently operating data replication engine replicates source database transactions including any new source database transactions committed in an audit trail of the source database before the currently operating data replication engine stops replicating. A list of active database transactions is generated and recorded as a first token in the audit trail. The currently operating data replication engine is stopped when all of the database transactions in the first token have completed. The new data replication engine is started and positioned to begin processing source database transactions in the audit trail of the source database at a beginning position that is located at or prior to the position of the first token in the audit trail. The new data replication engine replicates source database transactions by ignoring source database transactions listed in ...

MERGING DATA VOLUMES AND DERIVATIVE VERSIONS OF THE DATA VOLUMES

Disclosed are systems, methods, and software for performing version control. In a particular embodiment, a non-transitory computer readable medium is provided having stored therein program instructions that, when executed by a computer system, direct the computer system to perform a method of version control. The method includes executing a plurality of virtual machines from a plurality of derivative versions of an ancestor data volume, wherein the ancestor data volume and the plurality of derivative versions each comprise a plurality of files. The method further includes tracking modifications to the plurality of files in each of the plurality of derivative versions and merging the plurality of derivative versions with the ancestor data volume based on the modifications.

Initialization protocol for a peer-to-peer replication environment

Techniques for initializing a connection in a peer-to-peer database replication environment. A message is received from an initiator node, indicating that the initiator node will begin replicating changes made to a particular data entity. One or more replicated changes relating to the database entity are received from the initiator node. If an updated instance of the data entity does not currently exist on a target node then one or more load operations are performed to create the updated instance. The stored replicated changes may then be processed against the updated instance of the database entity.

Cluster-Wide Read-Copy Update System And Method

A system, method and computer program product for synchronizing updates to shared mutable data in a clustered data processing system. A data element update operation is performed at each node of the cluster while preserving a pre-update view of the shared mutable data, or an associated operational mode, on behalf of readers that may be utilizing the pre-update view. A request is made for detection of a grace period, and grace period detection processing is performed for detecting when the cluster-wide grace period has occurred. When it does, a deferred action associated with the update operation it taken, such as removal of a pre-update view of the data element or termination of an associated mode of operation.

File-backed in-memory structured storage for service synchronization

Providing synchronization to a local system. Embodiments may include downloading a batch of changes in a heterogeneous batch. The batch of changes is correlated to an anchor affiliated with a synchronization service. The anchor is a reference point indicating time or relative order. The batch of changes and the anchor are serialized to a non-volatile storage medium as a heterogeneous set. After serializing the batch of changes to a non-volatile storage medium as a heterogeneous set, entities in the batch of changes are parsed out into entities in in-memory representations. Similarly, embodiments may receive user input modifying a plurality of data entities, store on a non-volatile storage medium a serialized heterogeneous representation of the modified data entities, and upload the serialized heterogeneous representation to a synchronization service.

Method and apparatus for managing shared data at a portable electronic device of a first entity

A method and apparatus for managing shared data at a portable electronic device of a first entity is provided. A message is received advising that data associated with a second entity is being shared. A request is transmitted to a server for a list of shared folders associated with the second entity, in response to an option to view shared folders associated with the second entity being selected. The list is received. An initialize command is transmitted to the server, the initialize command identifying at least one folder in the list. The data associated with the second entity is received, responsive to the transmitting the initialize command. The data is stored in association with a second entity identifier.

Method and apparatus for providing address book restoration

An apparatus for providing address book restoration includes at least one processor and at least one memory including computer program code. The at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus to receive an instruction to restore a record associated with a contact previously deleted from each of at least two synchronized address books, and recover the record from a deleted file repository at which the record was stored prior to deletion of the record, the deleted file repository being a memory location different than the address books. A corresponding method and computer program product are also provided.

Middleware data log system

A method of maintaining a data store is disclosed. The data store is distributed across a plurality of storage provider instances, including a master storage provider instance and a plurality of additional storage provider instances. A master data log is updated to reflect a change to the data store. The master data log is synchronized with a plurality of replicas of the master data log. The change is propagated across the plurality of additional storage provider instances based on an analysis of the replicas.

Method, System, and Computer Program Product for Synchronization of Similar Data Objects with Event Information

A computer-implemented method for updating an object on a mobile device is provided. The computer-implemented method includes maintaining at a server a plurality of channels, the plurality of channels including at least a first representation of the object, generating a modification event when a change to the object is detected, responsive to the modification event, at least updating the representation of the object yielding a second representation of the object, and conveying, during a synchronization process, the second representation of the object to the mobile device.

Openstack database replication

Several different embodiments of a massively scalable object storage system are described. The object storage system is particularly useful for storage in a cloud computing installation whereby shared servers provide resources, software, and data to computers and other devices on demand. In several embodiments, the object storage system includes a ring implementation used to associate object storage commands with particular physical servers such that certain guarantees of consistency, availability, and performance can be met. In other embodiments, the object storage system includes a synchronization protocol used to order operations across a distributed system. In a third set of embodiments, the object storage system includes a metadata management system. In a fourth set of embodiments, the object storage system uses a structured information synchronization system. Features from each set of embodiments can be used to improve the performance and scalability of a cloud computing object storage system.

Delayed updating of shared data

To provide delayed updating of shared data, a concept of dualistic sequence information is introduced. In the concept, if during local modification of data, a modification to the data is published by another user, a local deviation is created, and when the modification is published, it is associated with an unambiguous sequence reference and the local deviation.

Low latency replication techniques with content addressable storage

A CAS data storage method and apparatus comprising: receiving input data including a succession of data items with corresponding logical addresses at a source CAS data storage space for storage therein and for replication at a destination CAS data storage space, generating a hash key for each data item at the source storage space, comparing respective hash keys with hash keys stored at a hash key storage table, to determine whether respective further data items are already present at the destination storage device; transferring respective data items to the destination storage space if no match is made to a hash key stored at the hash key storage table, but not transferring respective further data items if a match is made to a hash key stored at the hash key storage table, thereby transferring to the destination storage space only unique data items.

Transference control method, transference control apparatus and recording medium of transference control program

A transference control apparatus includes a communication controller, and a processor that executes a procedure, the procedure including processing a control that causes first processing, in which an update log representing the update content of updating performed for a first database is transmitted, by the communication controller, to a second database serving as a copy destination for the first database, to be performed with priority over second processing, in which transmission target data contained in the first database is transmitted to the second database, and restraining, in the case that updating of the transmission target data is allowed in the first database, the control that causes the first processing to be performed with priority.

Methods for tracking database changes and devices thereof

Methods, devices, and computer-readable storage media for tracking changes in a database including at least one database table include adding a first column to a schema of each database table. The first column includes a row update identifier for each row of the database table. A global update identifier is incremented in response to a row change and recorded as the row update identifier for the changed row. In response to a latest database synchronization event, a reference table including a synchronization update identifier is generated, and the global update identifier is incremented and recorded as the synchronization update identifier. An update to a row in a database table is tracked based at least upon determining that a row update identifier of the updated row in the database table is greater than the synchronization update identifier in the reference table.

Synchronization of database changes among multiple devices

Methods, systems, and apparatus, including computer program products, for synchronizing database changes among multiple devices. In one aspect, a method includes determining one or more primary differences between a primary database and a local synchronization database; receiving, from a secondary device, one or more secondary differences associated with a secondary database; merging the primary differences and the secondary differences to generate a set of merged differences; providing the set of merged differences to the secondary device; receiving, from the secondary device, an acknowledgment indicating receipt of the set of merged differences; and updating the primary database in accordance with the set of merged differences after receipt of the acknowledgment. Further, at least one conflict can be detected between the primary differences and the secondary differences, and can be resolved prior to generating the set of merged differences.

System and Method for Data Set Synchronization and Replication

According to one embodiment of the present disclosure, a method for synchronizing data sets includes receiving a request to synchronize a first data set associated with a first server and a second data set associated with a second server. The method also includes determining, with reference to one or more replication constraints, whether to begin synchronization. The method further includes applying one or more resource control actions in response to determining to begin synchronization.

Remote data collection systems and methods using read only data extraction and dynamic data handling

Remote data collection systems and methods retrieve data including financial, sales, marketing, operational and the like data from a plurality of databases and database types remotely over a network in an automated, platform-agnostic manner. A remote data collection system includes a network interface, a connection to a data source, a processor communicatively coupled to the network interface and the connection, and memory storing instructions for remote data collection that, when executed, cause the processor to: receive a request to extract data from the data source; extract the data in a non-intrusive manner from the data source using a two phase process comprising a reconciliation phase and a collection phase; and transmit one of an entire set and a subset of the extracted data based on the request.

Fast snapshots

A fast snapshot is configured to store a state of a computing environment at a point in time. The fast snapshot operation is performed by avoiding reference counts of one or more data units associated with the snapshot from being updated at a creation and a deletion time.

Data Archiving and De-Archiving in a Business Environment

A computer implemented method for data archiving and dearchiving in a business environment is described herein. According to an implementation, the method includes obtaining metadata configuration associated with a source database storing transaction information, the transaction information being related to a business transaction. Based on the metadata configuration an archive specification is generated. Further, based on the archive specification and the metadata configuration, an archive mapping is configured. According to an aspect, the archive mapping is synchronized with an update in one of the metadata configuration. The archive mapping can be implemented for archiving the transaction information. Additionally, the archived transaction information can be de-archived, or can be purged onto a back-up repository.

System and method for database synchronization for medical records

A system and method for performing database synchronization between a first database of a first device and a second database of a second device is disclosed. The first database stores medical records having non-time based counter values associated thereto. The second database stores medical records having timestamps associated thereto. The first device includes a first database synchronization module that maintains a last timestamp indicating a last medical record that was received from the second device. The first database synchronization module transmits, to the second device, a request for synchronization and the last timestamp. The second device includes a second database synchronization module that maintains a last counter value indicative of a last medical record that was received from the first device, and that transmits, to the first device, a second request for synchronization and the last counter value.

Information processing system, computer-readable non-transitory storage medium, information processing method and information processor

Each information processor executes a mission which allows participation of users of a plurality of information processors using wireless communication. While the mission is being executed, a mission beacon including personal score information is repeatedly transmitted to an unspecified apparatus from each of the information processors. Each information processor calculates a total score based on the personal score information included in the mission beacon of each of other information processors and determines whether the mission has been successful or not based on the total score.

Automatic conflict resolution

A method and apparatus is for implementing a conflict resolution policy. The method includes providing a rule set that includes a plurality of rules that define the conflict resolution policy. An algorithm is generated by decomposing each rule in the rule set into at least one pre-action that is to be performed on data in identified fields in first and second conflicting objects, at least one condition that is to be applied to the first and second conflicting objects, and at least one action that is to be performed on the identified fields in the first and second conflicting objects if the at least one condition is satisfied. The algorithm is loaded from a configuration file for execution during a synchronization component runtime using first and second conflicting objects provided by the synchronization component.

Asynchronous distributed object uploading for replicated content addressable storage clusters

A method performed by two or more devices of a group of devices in a distributed data replication system may include receiving a group of chunks having a same unique temporary identifier, the group of chunks comprising an object to be uploaded; creating an entry for the object in a replicated index, the entry being keyed by the unique temporary identifier, and the replicated index being replicated at each of the two or more devices; and determining, by an initiating device of the two or more devices, that a union of the group of chunks contains all data of the object. The method may also include calculating a content-based identifier to the object; creating another entry for the object in the replicated index, the other entry being keyed by the content-based identifier; and updating the replicated index to point from the unique temporary identifier to the content-based identifier.

Transitioning application replication configurations in a networked computing environment

Embodiments of the present invention provide an approach for providing non-disruptive transitioning of application replication configurations and proactive analysis of possible error scenarios. Specifically, under embodiments of the present invention, a common integration model (CIM)-compatible representation of a system replication plan is provided in a computer data structure. Based on the representation, a hierarchical tree data structure having a set of nodes is created. A set of system configuration updates pertaining to the set of nodes are then classified (e.g., based upon the type of configuration update). Once the set of nodes has been classified, the set of nodes may then be analyzed to determine if any nodes of the set are isomorphic. If so, the plan can be modified accordingly. In any event, the replication plan (or modified replication plan) may then be implemented.

System and method for providing intra-process communication for an application programming interface

Systems and methods are provided for intra-process communication in a backup framework. A custom requester is registered with a framework configured to run and coordinate one or more requesters and one or more providers, wherein the framework provides an application programmer interface for the requesters and providers. A first requester from the one or more requesters starts a backup process for a first data set associated with a first application. The first requester determines that the data storage device associated with the first data set and the first application is exported by a data management system. The first requester selects a first provider to create a backup of the first data set for the backup process, wherein the first requester and first provider are separate threads in a same process space that can communicate directly with each other using one or more intra-process communication channels.

Front end and backend replicated storage

An existing primary data copy can be maintained on an existing primary front end server node. One or more existing secondary data copies can be maintained on one or more existing secondary front end server nodes to minor the existing primary data copy (such as by making synchronous changes to the secondary data copies). One or more existing backup data copies can be maintained on an existing backend server node to mirror the existing primary data copy (such as by making asynchronous changes to the one or more backup data copies). The existing backend server node can be accessible by one or more of the front end nodes. In response to detecting a failure of the existing primary data copy, one of the one or more secondary data copies can be designated as a new primary data copy in place of the existing primary data copy.

Metadata-based Test Data Generation

Example embodiments disclosed herein relate to a method for generating test data for form validation, where the method includes maintaining metadata for a database, mapping a control field of the form to a column of the database, and generating test data based on the mapping and the metadata.

Cloud computing-based data sharing system and method

Aspects of the present disclosure relate to cloud computing-based data sharing system and method, the system including a plurality of industrial device management units configured to transmit a recent shared data to a cloud server by periodically communicating with the cloud server, to receive a recent shared data of other industrial device management units received from the cloud server and to synchronize the shared data by updating the recent shared data to its own shared data, and a cloud server configured to compare a received shared data with a pre-stored shared data, in a case a shared data is received form an industrial device management unit among the plurality of industrial device management units through a communication network, to update its shared data as a result of the comparison, and to transmit a notification message including the updated shared data to other industrial device management unit through the communication network.

Distributed information synchronization

Processing a prepared update is disclosed. A prepared update associated with a request that has been used by the sender to update a local version of a data associated with the sender is received from a sender. Based at least in part on an identifier included in the prepared update, a selected data handler is selected among a plurality of data handlers. The selected data handler is used to update a centralized version of the data at least in part by using the received prepared update. The centralized version of the data has been previously updated using a plurality of prepared updates received from a plurality of senders. The updated centralized version of the data is sent to update the local version of the data associated with the sender.

Role assignments in a cloud infrastructure

Provisioning, managing and tracking of services provided by a cloud infrastructure system are described. A subscription order request from a customer for services provided by the cloud infrastructure system is received. A separation of roles performed by different customer entities in a customer's organization when the customer subscribes to services in the cloud infrastructure system is determined. The roles relate to providing information about different portions of the subscription order by the customer entities. Upon completion of the assigned roles by the relevant customer entities, the subscription order is provisioned to the customer.

Systems and methods for selective data replication

Systems and methods for performing data replication are disclosed. Determining whether to update replicated data typically involves comparison of readily obtainable attributes of a given source file and its corresponding replicated file. Such attributes can be obtained from, for example, metadata. In certain situations, an additional assessment of the source and replicated files can be beneficial. For example, if integrity of an existing replicated file's content is maintained, one may not want to re-replicate the corresponding source file. For large source files, such a decision can provide substantial reductions in expenditures of available computing and network resources. In certain embodiments, a threshold for identifying such large files can be based on one or more operating parameters such as network type and available bandwidth. In certain embodiments, replication file's integrity can be checked by calculating and comparing checksums for the replication file and its corresponding source file.

Synchronization of sequential access storage components with backup catalog

Subsequent to a storage operation performed on the source instance by a source component, a synchronization message is sent to a replicated component for the replicated instance. The synchronization message is stored locally in a persistent storage location associated with the source component along with an indicator representative of a time the storage operation was performed. Pursuant to receipt of the synchronization message by the replicated component, the replicated component is updated to a dirty state to indicate a lack of full synchronization between the source and replicated instances.

Separation of pod provisioning and service provisioning

A method for POD provisioning and service provisioning is disclosed. The method may comprise storing, by a cloud infrastructure system, subscription order information from a customer identifying a service from a set of cloud services provided by the cloud infrastructure system, the cloud infrastructure system comprising one or more computing devices, wherein the subscription order information includes customer-specific configuration. Additionally, the method may comprise determining, by a computing device from the one or more computing devices, a service associated with the subscription order information. Moreover, the method may comprise mapping a pre-provisioned anonymous deployment to the subscription order information, wherein the pre-provisioned anonymous deployment is specifically pre-provisioned for the determined service. Furthermore, the method may comprise creating, by a computing device from the one or more computing devices, a service instance specifically for the customer by configuring the pre-provisioned anonymous deployment with the customer-specific configuration.

Device connection proxy through cloud to optimize network messages

Technology is disclosed herein for optimizing network connections using proxy connections. According to at least one embodiment, a computing device receives requests for network connections with remote servers from multiple computer applications running on the computing device. The computing device further establishes a network connection with a proxy server. The proxy server initiates and maintains proxy network connections with the remote servers. The proxy server can delay at least one network message of the network messages received from the remote servers such that the proxy server sends the network messages to the computing device in a batch. The proxy server batches the network messages such that the computing device stays at a high power consumption state for a time period shorter than an overall time period for which the computing device would stay at the high power consumption if the network messages were sent individually without batching.

Memory sharing across distributed nodes

A method and apparatus are disclosed for enabling nodes in a distributed system to share one or more memory portions. A home node makes a portion of its main memory available for sharing, and one or more sharer nodes mirrors that shared portion of the home node's main memory in its own main memory. To maintain memory coherency, a memory coherence protocol is implemented. Under this protocol, load and store instructions that target the mirrored memory portion of a sharer node are trapped, and store instructions that target the shared memory portion of a home node are trapped. With this protocol, valid data is obtained from the home node and updates are propagated to the home node. Thus, no “dirty” data is transferred between sharer nodes. As a result, the failure of one node will not cause the failure of another node or the failure of the entire system.

SYSTEMS AND METHODS FOR FILTERING ELECTRONIC ACTIVITIES BY PARSING CURRENT AND HISTORICAL ELECTRONIC ACTIVITIES

The present disclosure relates to systems and methods for filtering electronic activities. The method includes identifying an electronic activity. The method includes parsing the electronic activity to identify one or more electronic accounts in the electronic activity. The method includes determining, responsive to parsing the electronic activity, that the electronic activity is associated with an electronic account of the one or more electronic accounts. The method includes selecting, based on the electronic account, one or more filtering policies associated with the data source provider to apply to the electronic activity. The method includes determining, by applying the selected one or more filtering policies to the electronic activity, to restrict the electronic activity from further processing based on the electronic activity satisfying at least one of the selected one or more filtering policies. The method includes restricting, the electronic activity from further processing. 1. A method comprising:identifying, by one or more processors, an electronic activity associated with a data source provider;parsing, by the one or more processors, the electronic activity to identify one or more electronic accounts in the electronic activity;determining, by the one or more processors, responsive to parsing the electronic activity, that the electronic activity is associated with an electronic account of the one or more electronic accounts, the electronic account corresponding to the data source provider;selecting, by the one or more processors based on the electronic account, one or more filtering policies associated with the data source provider to apply to the electronic activity, the selected one or more filtering policies including at least one of i) a keyword policy configured to restrict electronic activities including a predetermined keyword; ii) a regex pattern policy configured to restrict electronic activities including one or more character strings that match ...

Multi-stage slice recovery in a dispersed storage network

A method for use by a computing device in a dispersed storage network (DSN) to recover corrupt encoded data slices. In response to a request to storage units of the DSN for encoded data slices corresponding to a data segment, the computing device of a receives less than a decode threshold number of valid encoded data slices and at least one integrity error message that provides an indication of a corrupt encoded data slice. The computing device requests and receives at least one corrupt encoded data slice corresponding to the integrity error message(s). Utilizing at least one correction approach involving stored integrity data, the computing device then corrects the corrupt slice(s) to produce a decode threshold number of encoded data slices in order to decode the corresponding data segment. A variety of correction approaches may be employed, including a multi-stage approach that utilizes data from both valid and invalid slices.

List request processing during a dispersed storage network configuration change

A method includes identifying resources that are affiliated with a range of slice names of a listing request. The method further includes creating an ordered classification of the resources. The method further includes a resource determining whether it is in a last class of the ordered classification. When the resource is in the last class, it processes the listing request to generate a listing response and sends the listing response to another resource in a next lower class. When the resource is not in the last class, it identifies a second resource for proxying of the listing request and sends the listing request to the second resource. The method further includes receiving a cumulated listing response from the second resource. The method further includes processing the listing request to generate the listing response. The method further includes combining the listing response with the cumulated listing response.

Using distributed source control in a centralized source control environment

A method is presented for using a distributed source control system with a centralized source control system. A first set of files is obtained from a source control repository and stored on a first electronic computing device. The first set of files comprises all or part of a code base in the centralized source control system. A request is received for at least part of the code base from a second electronic computing device in a distributed source control system. As a result of the request, at least a part of the first set of files is sent to the second electronic computing device. A change set for the first set of files is received from the second electronic computing device. The change set is processed to be in a format compatible with the source control repository. The change set is submitted to the source control repository.

MULTI-MASTER DATA REPLICATION IN A DISTRIBUTED MULTI-TENANT SYSTEM

A multi-master replication system is disclosed. The multi-master replication system allows a large set of peer instances to collaboratively replicate data to each other. According to an example, a change detection thread running on a first server associated with a first instance of multiple instances of a replicated database monitors for changes to any of multiple records within one or more shared tables of the replicated database. Responsive to detection of a change to a record, an item is stored by the change detection thread onto a queue containing information regarding the change. Groups of changes are packaged into multiple chunks, in which each chunk (i) corresponds to a discrete unit of progress for both change detection and transport; (ii) is associated with multiple changed records; (iii) contains metadata about the multiple changed records; and (iv) does not contain data from the one or more shared tables. 1. A method comprising:monitoring, by a change detection thread running on a first server associated with a first instance of a plurality of instances of a replicated database, for changes to any of a plurality of records within one or more shared tables of the replicated database;responsive to detection of a change to a record of the plurality of records, storing, by the change detection thread, an item onto a queue containing information regarding the change; and corresponds to a discrete unit of progress for both change detection and transport;', 'is associated with a plurality of changed records;', 'contains metadata about the plurality of changed records; and', 'does not contain data from the one or more shared tables., 'packaging groups of changes into a plurality of chunks, wherein each chunk of the plurality of chunks2. The method of claim 1 , wherein said packaging comprises:periodically aggregating, by an indexer thread running on the first server, information regarding the plurality of changed records that have been committed since a previous ...

Pipelining Paxos State Machines

Paxos transactions are pipelined in a distributed database formed by a plurality of replica servers. A leader server is selected by consensus of the replicas, and receives a lock on leadership for an epoch. The leader gets Paxos log numbers for the current epoch, which are greater than the numbers allocated in previous epochs. The leader receives database write requests, and assigns a Paxos number to each request. The leader constructs a proposed transaction for each request, which includes the assigned Paxos number and incorporates the request. The leader transmits the proposed transactions to the replicas. Two or more write requests that access distinct objects in the database can proceed simultaneously. The leader commits a proposed transaction to the database after receiving a plurality of confirmations for the proposed transaction from the replicas. After all the Paxos numbers have been assigned, inter-epoch tasks are performed before beginning a subsequent epoch.

PREDICTIVE AND ADAPTIVE QUEUE FLUSHING FOR REAL-TIME DATA RECONCILIATION BETWEEN LOCAL AND REMOTE DATABASES

Predictive queue flushing for real-time synchronization of data sets between two data stores, comprising a data synchronization software module that interfaces with each data store, and uses a queue monitor to record and store changes to data on each data store and calculate velocity and acceleration of event arrivals, and a policy manager to manage synchronization, and a query generator to incorporate policies from the policy manager and measurements from the queue monitor to direct the data synchronization software module, flushing the change queue in accordance with the established synchronization policy, yielding synchronized shared data sets. 1. A system for real-time synchronization of data between disparate cloud data sources , comprising: receive and store data from a plurality of sources, wherein the plurality of sources comprises at least a second cloud-based database of a different design from that of the first cloud-based database; and', 'retrieve and supply data to a plurality of destinations, wherein at least one of the plurality of destinations is the second cloud-based database;, 'a first cloud-based database comprising at least a processor, a memory, and a plurality of programming instructions stored in the memory and operating on the processor, wherein the programming instructions, when operating on the processor, cause the processor to receive data records from a data source, wherein the data source comprises either the first cloud-based database or the second cloud-based database;', 'apply a modification step drawn from a set comprising field name changes, table name changes, data type changes, metadata storage design changes, and table assignment changes for translation of data between the data source and a data destination, wherein the data destination comprises either the first cloud-based database or the second cloud-based database; and', 'output modified data records to the data destination;, 'a data conversion module comprising at least a ...

Method of storing encoded data slices using a distributed agreement protocol

A method includes encoding a data object in accordance with dispersed storage error encoding parameters to produce sets of encoded data slices having sets of slice names. The dispersed storage error encoding parameters includes a pillar width number of encoded data slices. The data object is associated with a unique source name and each slice name includes a reference to the unique source name. The method further includes executing a distributed agreement protocol using the unique source name and coefficients regarding a plurality of storage units of the dispersed storage network (DSN) to produce a ranking of the plurality of storage units. The method further includes identifying the pillar width number of storage units of the plurality of storage units based on the ranking of the storage units. The method further includes sending the plurality of sets of encoded data slices to the pillar width number of storage units for storage therein.

Fast snapshots

A fast snapshot is configured to store a state of a computing environment at a point in time. The fast snapshot operation is performed by avoiding reference counts of one or more data units associated with the snapshot from being updated at a creation and a deletion time.

DATA REPLICATION AND DATA FAILOVER IN DATABASE SYSTEMS

Replication and failover of database data is disclosed. A method includes replicating database data stored in a primary deployment such that the database data is further stored in a secondary deployment. The method includes executing one or more updates to the database data at the secondary deployment when the primary deployment is unavailable and propagating the one or more updates to the primary deployment when the primary deployment becomes available again. The method includes executing queries on the database data at the primary deployment when the primary deployment is available. 1. A system comprising:means for replicating database data stored in a primary deployment such that the database data is further stored in a secondary deployment;means for determining that the primary deployment is unavailable;means for executing one or more transactions on the database data at the secondary deployment in response to determining that the primary deployment is unavailable;means for determining that that the primary deployment is no longer unavailable;means for propagating the one or more transactions on the database data to the primary deployment in response to determining that the primary deployment is no longer unavailable; andmeans for executing queries on the database data at the primary deployment when the primary deployment is available.2. The system of claim 1 , further comprising means for executing new transactions on the database data at the primary deployment and the secondary deployment when each of the primary deployment and the secondary deployment is available.3. The system of claim 1 , further comprising means for shifting execution of queries on the database data from the primary deployment to the secondary deployment for a duration of time the primary deployment is unavailable.4. The system of claim 1 , wherein the primary deployment and the secondary deployment are located in different geographic locations.5. The system of claim 1 , wherein the ...

CREATING REPLICAS AT USER-DEFINED POINTS IN TIME

Example embodiments of the present invention relate to methods, systems, and computer program products for determining replicas for removal from a plurality of replicas as defined in a data retention policy. The method includes performing asynchronous replication from a source to a target according to time-based replication cycles. A retention policy then may be identified defining a number of asynchronous replication replicas to retain and a plurality of associated periods which then may be retained according to the policy. 1. A computer-implemented method comprising:performing a first replication from a source to a target according to a time-based replication cycle to generate a first replica;incrementing a value of a counter corresponding to a first replica retention period;determining that the value of the counter is not equal to or a multiple of a predetermined threshold value associated with a second replica retention period;marking the first replica as belonging to the first replica retention period based at least in part on the determination that the value of the counter is not equal to or a multiple of the predetermined threshold value associated with the second replica retention period;performing a second replication from the source to the target according to the time-based replication cycle to generate a second replica;incrementing the value of the counter corresponding to the first replica retention period;determining that the value of the counter is equal to or a multiple of the predetermined threshold value associated with the second replica retention period; andmarking the second replica as belonging to the second replica retention period based at least in part on the determination that the value of the counter is equal to or a multiple of the predetermined threshold value associated with the second replica retention period.2. The computer-implemented method of claim 1 , further comprising:resetting the value of the counter based at least in part on ...

Database access performance improvement

In an approach for database log management and log access performance improvement, a processor receives an access request for an optimal control file to minimize a database access performance impact. A processor determines the optimal control file based on usage of a plurality of control files for a database. A processor outputs the optimal control file to a user to access the database. A processor provides an incremental sub-control file of the optimal control file to the user. The incremental sub-control file is an incremental view of the optimal control file to query the incremental log data in the database.

Detecting data skew in a join operation

Systems, methods, and devices, for managing data skew during a join operation are disclosed. A method includes computing a hash value for a join operation and detecting data skew on a probe side of the join operation at a runtime of the join operation using a lightweight sketch data structure. The method includes identifying a frequent probe-side join key on the probe side of the join operation during a probe phase of the join operation. The method includes identifying a frequent build-side row having a build-side join key corresponding with the frequent probe-side join key. The method includes asynchronously distributing the frequent build-side row to one or more remote servers.

Resource provisioning systems and methods

A method and apparatus managing a set of processors for a set of queries is described. In an exemplary embodiment, a device receives a set of queries for a data warehouse, the set of queries including one or more queries to be processed by the data warehouse. The device further provisions a set of processors from a first plurality of processors, where the set of processors to process the set of queries, and a set of storage resources to store data for the set of queries. In addition, the device monitors a utilization of the set of processors as the set of processors processes the set of queries. The device additionally updates a number of the processors in the set of processors provisioned based on the utilization/Furthermore, the device processes the set of queries using the updated set of processors.

METHOD AND SYSTEM FOR DATA HANDLING

A method of transferring data between database locations comprising: providing a location reference comprising location values of one or more data collections stored in a database; providing an indicator in association with at least one data collection of the one or more data collections, the indicator being provided in response to initiation of a transfer process of the at least one data collection from a first location of the database to a second location of the database; wherein the indicator is changeable between a first state and a second state, the first state being indicative of the initiation of the transfer process; and as part of the transfer process of the at least one data collection, selectively updating a location value of the at least one data collection in the location reference, in dependence on a determination of whether the indicator is in the first state or the second state. 1. A method of migrating data between a first database location and a second database location of a computer implemented database , the method comprising:storing in at least one memory in at least one server, a look-up table comprising respective location values of a plurality of data collections stored in said computer implemented database, wherein a respective location value indicates a respective first location within the computer implemented database where an associated data collection is stored, wherein when access to a respective data collection is required, a look up request is performed with respect to the look up table for the respective location value for said respective data collection;providing at least a part of the look-up table to a first cache;storing by at least one processor, in the look-up table which is stored in the at least one memory in said at least one server, an indicator for each of said data collections being migrated in response to initiation of a respective data migration process of one or more of said data collections from a respective first ...

CENTRALIZED DATABASE SYSTEM WITH GEOGRAPHICALLY PARTITIONED DATA

A computer-implemented method for storing documents in a centralized database system based on geographical data localization includes receiving, by a database manager, a request to update a document according to a data localization criterion, the document stored in a local storage device of the centralized database system. The method further includes determining, by the database manager, a geographical location for a local copy of the document based on an attribute of the document matching the localization criterion. The method further includes creating, by the database manager, a remote copy of the document on a storage device that is physically located in the geographical location. The method further includes modifying, by the database manager, the local copy of the document to include a location information of the remote copy. 1. A computer-implemented method for storing documents in a centralized database system based on geographical data localization , the method comprising:receiving, by a database manager, a request to create a document according to a data localization criterion;creating, by the database manager, the document in a local storage device of the centralized database system;determining, by the database manager, a geographical location for the document based on an attribute of the document matching the localization criterion;creating, by the database manager, a remote copy of the document on a storage device that is physically located in the geographical location; andmodifying, by the database manager, the local copy of the document to include a location information of the remote copy, wherein the database manager asynchronously copies application data from the local copy to the remote copy before modifying the local copy.2. The method of claim 1 , wherein the geographical location of the storage device is distinct from a geographical location of the local storage device of the centralized database system.3. The method of claim 1 , wherein modifying ...

Cross-acl multi-master replication

Techniques for cross-ACL multi-master replication are provided. The techniques allow a replication site in a multi-master replication system implementing an asynchronous replication protocol and an access control policy to appropriately apply received data change updates to data maintained at the site even where a data change update is missing information because of the implemented access control policy.

SYSTEMS AND METHODS FOR IDENTIFYING A SEQUENCE OF EVENTS AND PARTICIPANTS FOR RECORD OBJECTS

Methods, systems, and storage media for identifying a sequence of events and participants for record objects are disclosed. Exemplary implementations may: access record objects of a system of record; identify a subset of record objects associated with a group entity and having a first record object status; identify one or more electronic activities linked to the record objects; determine an event-participant pattern based on the electronic activities linked to the record object; identify electronic activities linked with a second record object; determine that a first event is performed by the a participant type and a second event is not yet performed by a second participant type; generate a content item identifying an action to trigger a performance of the second event; and transmit the content item to a device of a participant of at least one electronic activity linked with the second record object. 1. A method , comprising:accessing, by one or more processors, a plurality of record objects of a system of record of a data source provider, each record object of the plurality of record objects corresponding to a respective group entity, each record object comprising one or more object field-value pairs and linked to one or more electronic activities;identifying, by the one or more processors, a subset of record objects of the plurality of record objects associated with a group entity and having a first record object status;identifying, by the one or more processors, for each record object of the subset of record objects, one or more electronic activities linked to the record object, each electronic activity identifying one or more participants and corresponding to at least one event;determining, by the one or more processors, for each record object of the subset of record objects, an event-participant pattern based on the electronic activities linked to the record object, the event-participant pattern including at least a first event performed by a first participant ...

BILATERAL ASSERTION MODEL AND LEDGER IMPLEMENTATION THEREOF

A computer implemented method for implementing a real time reconciling shared data structure is disclosed. The shared data structure may be stored in a memory, and a portion of the shared data structure may be coupled with a processor. The computer implemented method may be used for interacting with the shared data structure. The method may comprise the exchange of messages, requesting to change data in the shared data structure, between selected participants, wherein some of the participants must validate requested changes to the shared data. If all participants validate the requests to change data the changes to the data are made, and if less than all participants validate the requests then the changes are not made. 1. A computer implemented method for implementing a real time reconciling shared data structure among a plurality of participants , the shared data structure being stored in a memory , a portion of the shared data structure being coupled with a processor , the method comprising:receiving, by the processor via a network interface, a request data transaction message from a participant of the plurality of participants, the request data transaction message comprising data indicative of a request by the participant to modify data stored in the portion of the shared data structure;validating, by the processor, the request data transaction message using one or more rules stored in memory and related to the shared data structure;identifying, by the processor, based on the request data transaction message, at least one other participant of the plurality of participants to validate modifications to the data;receiving, by the processor via the network interface validation data transaction messages from each of the identified at least one other participants, each of the received validation data transaction messages comprising data indicative of validation of the request to modify the data in the portion of the shared data structure by the at least one other ...

EFFICIENT RECONSTRUCTION OF A DEDUPLICATION DATABASE

If a deduplication database becomes corrupted or is lost, the deduplication database may be reconstructed by restoring an earlier backup copy of the deduplication database. The backup copy of the deduplication database, however, may not be synced with the deduplicated data blocks that are presently stored in the secondary storage subsystem. To address the possibility of data loss and improve reconstruction timing, a deduplicated storage system is provided according to certain embodiments that uses one or more mechanisms to restore the deduplication database and resync with the secondary storage by using one or more journal files that track the data blocks that have been or may have been deleted from the secondary storage since the last deduplication database backup. 1. An information management system configured to restore a deduplication database , the information management system comprising: receive a request to reconstruct a deduplication database (DDB) to an earlier version;', 'retrieve a first DDB backup, wherein the first DDB backup is a backup of the DDB created at a first time;', 'restore the first DDB backup to a DDB media agent;', 'determine a corresponding zero-reference file, wherein the zero-reference file includes one or more identifications of removed data blocks that have been deleted or have been marked for deletion from one or more secondary storage devices; and', wherein the primary table identifies data blocks stored in a secondary storage device and data chunks associated with the data blocks, and', 'wherein the primary table for each identified data block comprises at least a primary identifier, a unique signature, and a flag., 'for at least one identification of the removed data blocks in the zero-reference file, modify a flag associated with the at least one identification of the removed data blocks in a primary table of the restored first DDB to indicate that the removed data blocks may no longer be stored in the one or more secondary ...

Subscription updates in multiple device language models

Systems and methods for intelligent language models that can be used across multiple devices are provided. Some embodiments provide for a client-server system for integrating change events from each device running a local language processing system into a master language model. The change events can be integrated, not only into the master model, but also into each of the other local language models. As a result, some embodiments enable restoration to new devices as well as synchronization of usage across multiple devices. In addition, real-time messaging can be used on selected messages to ensure that high priority change events are updated quickly across all active devices. Using a subscription model driven by a server infrastructure, utilization logic on the client side can also drive selective language model updates.

System for live-migration and automated recovery of applications in a distributed system

A method and apparatus for distribution of applications amongst a number of servers, ensuring that changes to application data on a master for that application are asynchronously replicated to a number of slaves for that application. Servers may be located in geographically diverse locations; the invention permits data replication over high-latency and lossy network connections and failure-tolerance under hardware and network failure conditions. Access to applications is mediated by a distributed protocol handler which allows any request for any application to be addressed to any server, and which, when working in tandem with the replication system, pauses connections momentarily to allow seamless, consistent live-migration of applications and their state between servers. Additionally, a system which controls the aforementioned live-migration based on dynamic measurement of load generated by each application and the topological preferences of each application, in order to automatically keep servers at an optimum utilisation level.

System for live-migration and automated recovery of applications in a distributed system

A method and apparatus for distribution of applications amongst a number of servers, ensuring that changes to application data on a master for that application are asynchronously replicated to a number of slaves for that application. Servers may be located in geographically diverse locations; the invention permits data replication over high-latency and lossy network connections and failure-tolerance under hardware and network failure conditions. Access to applications is mediated by a distributed protocol handler which allows any request for any application to be addressed to any server, and which, when working in tandem with the replication system, pauses connections momentarily to allow seamless, consistent live-migration of applications and their state between servers. Additionally, a system which controls the aforementioned live-migration based on dynamic measurement of load generated by each application and the topological preferences of each application, in order to automatically keep servers at an optimum utilisation level.

VERSIONED HIERARCHICAL DATA STRUCTURES IN A DISTRIBUTED DATA STORE

A distributed data store may maintain versioned hierarchical data structures. Different versions of a hierarchical data structure may be maintained consistent with a transaction log for the hierarchical data structure. When access requests directed to the hierarchical data structure are received, a version of the hierarchical data structure may be identified for processing an access request. For access requests with snapshot isolation, the identified version alone may be sufficient to consistently process the access request. For access requests with higher isolation requirements, such as serializable isolation, transactions based on the access request may be submitted to the transaction log so that access requests resulting in committed transactions may be allowed, whereas access requests resulting in conflicting transactions may be denied. 120-. (canceled)21. A system , comprising:a plurality of storage nodes, comprising a processor and a memory, that store respective versions of a hierarchical data structure consistent with a transaction log for the hierarchical data structure;a transaction log store, that maintains a transaction log for the hierarchical data structure; receive a request to commit a transaction to a hierarchical data structure that includes a transaction state generated by another one of the hierarchy storage nodes based one or more prior operations performed by the other one hierarchy storage node;', 'request a conflict validation for the transaction including the one or more prior operations from the transaction log store; and', 'responsive to determining that the transaction does not conflict based on the conflict validation received from transaction log store, return a response to the request acknowledging that the transaction has been committed to the hierarchical data structure., 'one of the storage nodes, configured to22. The system of claim 21 , wherein the request to commit the transaction includes one or more additional operations to ...

ORG SYNC ASYNC SUBSCRIBE EMAILS

Disclosed are some implementations of systems, apparatus, methods and computer program products for performing an asynchronous subscription process that enables a secondary organization to subscribe to receive data from a primary organization. A primary organization provides publish data indicating fields to which a secondary organization can subscribe. The secondary organization provides subscription data that indicates published fields to which the secondary organization has subscribed. In some implementations, a state of a connection between the primary organization and the secondary organization is maintained and updated to ensure that the publish data and subscription data cannot be modified until the subscription process is completed. 1. A system comprising:a database system implemented using a server system, the database system configurable to cause:providing, for presentation on a display of a user device of a second entity, publish metadata indicating a first set of object types, the publish metadata indicating data items stored in at least one database of a first entity and available via subscription from the first entity;processing, by the second entity, an indication of user input submitted at the user device, the user input selecting subscription metadata, the subscription metadata indicating at least a subset of the first set of object types;transmitting, by the second entity to the first entity, a request pertaining to a data subscription between the first entity and the second entity;updating, by the second entity, a second transaction state to indicate that a subscription is in progress, wherein the subscription metadata cannot be modified while the second transaction state indicates that a subscription is in progress; andafter updating the second transaction state and transmitting the request, initiating a subscription process to provide the subscription metadata to the first entity.2. The system as recited in claim 1 , the database system further ...

System for Automated Data Engineering for Large Scale Machine Learning

Accordingly, a data engineering system for machine learning at scale is disclosed. In one embodiment, the data engineering system includes an ingest processing module having a schema update submodule and a feature statistics update submodule, wherein the schema update submodule is configured to discover new features and add them to a schema, and wherein the feature statistics update submodule collects statistics for each feature to be used in an online transformation, a record store to store data from a data source, and a transformation module, to receive a low dimensional data instance from the record store and to receive the schema and feature statistics from the ingest processing module, and to transform the low dimensional data instance into a high dimensional representation. One embodiment provides a method for data engineering for machine learning at scale, the method including calling a built-in feature transformation or defining a new transformation, specifying a data source and compressing and storing the data, providing ingest-time processing by automatically analyzing necessary statistics for features, and then generating a schema for a dataset for subsequent data engineering. Other embodiments are disclosed herein. 1. A method for data engineering for machine learning at scale , the method comprising:calling a built-in feature transformation or defining a new transformation;specifying a data source and compressing and storing the data;providing ingest-time processing by automatically analyzing necessary statistics for features, and then generating a schema for a dataset for subsequent data engineering.2. The method of claim 1 , further comprising the feature transformation connecting to models trained by upstream training systems claim 1 , and using the trained models to generate new features.3. The method of claim 2 , further comprising selectively caching features for better efficiency in a subsequent transformation.4. The method of claim 1 , wherein ...

UNIVERSAL AUTOMATIC DATA UPDATE DETECTION AND PUBLICATION

Universal automatic data update detection and publication may include receiving a request for requested information, which includes an information element associated with a structured data location in a structured data storage unit, identifying a predicate referencing the structured data location, including the predicate in a predicates registry by compiling the predicate into an intermediate representation, creating an asynchronous message bus channel corresponding to the predicate, detecting a data operation for the structured data storage unit, wherein the information element is associated with the data operation, evaluating the intermediate representation, determining a predicate domain change type corresponding to the data operation, generating a change notification including an indication of the change type, and publishing the change notification to the asynchronous message bus channel such that a presentation via the client device is updated in response to the change notification. 1. A system , comprising:a processor; and receiving, from a display device, a request for data to be displayed by the display device;', 'transmitting, to the display device, the requested data, wherein the display device is configured to display the requested data;', storing a predicate associated with the requested data in a predicates registry;', 'creating an asynchronous message bus channel corresponding to the predicate and subscribing the display device to the asynchronous message bus channel after storing the predicate in the predicates registry;', 'detecting a data operation configured to modify the requested data;', 'determining a type of modification to the requested data based on whether a value of the predicate matches the data operation before modification of the requested data by the data operation and the value of the predicate does not match the data operation after modification of the requested data by the data operation; and', 'publishing a notification to the ...

SYSTEM AND METHOD FOR CLONING AS SQL SERVER AG DATABASES IN A HYPERCONVERGED SYSTEM

A system and method include creating, by an Availability Group (“AG”) controller in a virtual computing system, a first AG clone from a source database. The source database is stored on a primary replica node of an AG of the virtual computing system. The system and method also include creating, by the Controller, a second AG clone from the first AG clone and storing, by the Controller, the second AG clone on a secondary replica node of the AG. The second AG clone has a size of substantially zero. 1. A method comprising:creating, by a processor executing computer-readable instructions stored on a memory of a virtual computing system, a first Availability Group (AG) clone from a source database, wherein the source database is stored on a primary replica node of an AG of the virtual computing system;creating, by the processor, a second AG clone from the first AG clone; andstoring, by the processor, the second AG clone on a secondary replica node of the AG,wherein the second AG clone has a size of substantially zero.2. The method of claim 1 , wherein the first AG clone is stored on the primary replica node of the AG.3. The method of claim 1 , wherein creating the second AG clone from the first AG clone comprises:capturing, by the processor, a snapshot of the first AG clone; andcreating, by the processor, a linked cloned disk from the snapshot.4. The method of claim 3 , further comprising:capturing, by the processor, an empty log backup of the first AG clone; andapplying, by the processor, the empty log backup to the linked cloned disk for creating the second AG clone.5. The method of claim 1 , wherein multiple copies of the second AG clone are stored on the secondary replica node.6. The method of claim 1 , wherein the size of the first AG clone is greater than the size of the second AG clone.7. The method of claim 1 , wherein each of the first AG clone and the second AG clone comprises a linked cloned disk claim 1 , and wherein the size of the linked cloned disk is ...

METHOD FOR LOADING DATA IN A TARGET DATABASE SYSTEM

The present disclosure relates to a computer implemented method for loading data in a target database system. The method comprises: determining that a load of a source table is expected to occur in the target database system. A future target table may be provided in advance in accordance with a defined table schema, and thereafter a load request for loading the source table may be received. Data of the source table may be loaded into the future target table. 1. A computer implemented method for loading data in a target database system , the method comprising:determining, by one or more computer processors, that a loading of a source table is expected to occur in the target database system;providing, by one or more computer processors, a future target table in advance in accordance with a defined table schema;receiving, by one or more computer processors, a load request for loading the source table; andloading, by one or more computer processors, data of the source table into the future target table.2. The computer implemented method of claim 1 , wherein determining that the loading is expected to occur is performed in response to:creating, by one or more computer processors, the source table in a source database system, wherein the source and target database systems are configured to synchronize data between each other, wherein the source table has the defined table schema.3. The computer implemented method of claim 1 , wherein determining claim 1 , by one or more computer processors claim 1 , that the loading is expected to occur comprises:processing, by one or more computer processors, a historical dataset indicative of a history of data loading into the target database system, the historical dataset comprising entries indicating source tables and a time at which said loading was performed, and based on the processing, determining, by one or more computer processors, that the loading is expected to occur.4. The computer implemented method of claim 3 , wherein the ...

Data System on a Module (DSoM) for Connecting Computing Devices and Cloud-based Services

A communication device (e.g., a data system on a module (DSoM)/a Data System in a Package (DSiP)) for communicatively coupling a computing device with a cloud-based service to synchronize one or more data modifications, on an asynchronous basis with respect to one another is provided. The communication device may be configured to be communicatively coupled to the computing device and may include a wireless cellular transceiver. The communication device may be configured to one or more of transmit at least one data object from the computing device to the cloud-based service and receive at least one data object from the cloud-based service. Embodiments of the present disclosure may provide a distributed replicated spatiotemporal database packaged on a communication device and integrated with an internet-based secure communication hub. 1. A communication device for communicatively coupling a computing device with a cloud-based service to synchronize one or more data modifications , on an asynchronous basis with respect to one another , the communication device configured to be communicatively coupled to the computing device and comprising:a wireless cellular transceiver; [ receiving at least one data object from the computing device,', 'modifying at least one data object array of one or more data objects stored in memory of the communication device based upon, at least in part, the received at least one data object,', 'asynchronously, with respect to the computing device, transmitting, via the wireless cellular transceiver, at least a portion of the modified at least one data object array to the cloud-based service; and, 'transmit one or more data objects from the computing device to the cloud-based service, including, asynchronously, with respect to the computing device, receiving, via the wireless cellular transceiver, at least a portion of the at least one data object array from the cloud-based service,', 'modifying the one or more data objects of the at least one ...

COLLABORATIVE SPREADSHEET DATA VALIDATION AND INTEGRATION

A system comprising a computer-readable storage medium storing at least one program and a method for integrating collaborative spreadsheet data into one or more network applications is presented. Spreadsheet data and an application data schema are accessed. The spreadsheet data corresponds to a spreadsheet having one or more validation rules. The application data schema comprises a set of constraints on application data consumed by an application hosted by an application server. A notification is displayed in response to determining that the one or more validation rules exclude at least one constraint from the set of constraints. The notification includes an indication that the application data is unable to be synchronized with the spreadsheet data and specifies one or more additional validation rules to associate with the spreadsheet to trigger synchronization of the application data with the spreadsheet data. 120.-. (canceled)21. A system comprising:one or more processors of a machine; anda memory storing instructions that, when executed by at least one processor among the one or more processors, causes the machine to perform operations comprising:accessing spreadsheet data of a spreadsheet having one or more data validation rules;accessing an application data schema comprising a set of constraints on application data consumed by an application from among a plurality of applications hosted by one or more application servers;determining whether the one or more data validation rules include the set of constraints; andin response to determining the one or more data validation rules exclude at least one constraint from the set of constraints, causing display-of a notification that includes an indication that the application data is unable to be synchronized with the spreadsheet data, the notification specifying one or more additional validation rules to associate with the spreadsheet to trigger synchronization of the application data with the spreadsheet data, the one ...

Asynchronous cache coherency for mvcc based database systems

A database system comprises a persistent storage device, a log node including a memory and a processor, and a plurality of database nodes. A database node includes a cache memory configured to store a database instance, and a processor configured to initiate a database transaction by sending a snapshot request to the log node, the snapshot request including a list of pages that were either replaced or newly loaded in the cache memory. The log node processor is configured to send a snapshot response to the database node, wherein the snapshot response includes a snapshot of the database and a list of changed pages of the database instances. The database node processor is configured to update the status of the pages in cached memory according to the snapshot response and perform the database transaction.

PROJECTIONS FOR BIG DATABASE SYSTEMS

A database system comprised of a decoupled compute layer and storage layer is implemented to store, build, and maintain a canonical dataset, a temporary buffer, and projection datasets. The canonical dataset is a set of batch updated data. The data is appended in chunks to the canonical dataset such that the canonical dataset becomes a historical dataset over time. The buffer is a write ahead log that contains the most recent chunks of data and provides atomicity and durability for the database system. The projection datasets are indexes of the canonical dataset and/or the buffer that may have single or multiple column sort-orders and/or particular data formats. The writes to the canonical dataset, projection datasets, and buffer may be asynchronous and therefore the database system is advantageously less resource constrained. 1. A system comprising:{'claim-text': ['program instructions;', 'a canonical dataset;', 'at least a first projection dataset; and', 'a buffer; and'], '#text': 'one or more computer readable storage mediums configured to store:'}{'claim-text': ['receive a first data chunk comprising an edit to the canonical dataset;', 'temporarily store the first data chunk in the buffer;', {'claim-text': ['update the canonical dataset based on the first data chunk; and', 'update the first projection dataset based on at least a part of the first data chunk; and'], '#text': 'asynchronously:'}, 'flush the first data chunk from the buffer.'], '#text': 'one or more processors configured to execute the program instructions to cause the system to:'}2. The system of claim 1 , wherein the first projection dataset comprises a first sorted subset of data of the canonical dataset.3. The system of claim 2 , wherein the one or more computer readable storage mediums are further configured to store:a second projection dataset comprising a second sorted subset of data of the canonical dataset, wherein the second sorted subset of data is different from the first sorted subset ...

DATABASE MANAGEMENT SYSTEM DATA REPLICATION

In response to identifying an update operation for a system table of a DBMS, a row that corresponds to the update operation is identified using a partial image of the update operation. A before image that corresponds to the row is gathered and a new before image of the row that reflects the update operation is generated by overlaying the before image with the partial image. In response to identifying a delete operation that relates to the system table, a row of the system table that corresponds to the delete operation is identified using an identifier of the delete operation. A before image that corresponds to this row is gathered. A new before image of the row of the system table that reflects the delete operation is generated by applying the delete operation to this before image. 1. A computer-implemented method comprising:{'claim-text': ['identifying a delete operation as recorded in a roll-forward recovery log for the DBMS, wherein the delete operation relates to a system table that corresponds to the application data table;', 'identifying, using an identifier of the delete operation as stored within the roll-forward recovery log, a row of the system table that corresponds to the delete operation;', 'gathering a before image that corresponds to the row of the system table; and', 'generating a new before image of the row of the system table that reflects the delete operation by applying the delete operation to the before image.'], '#text': 'enabling replication of an application data table of a database management system (DBMS) by:'}2. The computer-implemented method of claim 1 , wherein the row is a first row and the enabling replication of the application data table of a database management system (DBMS) further comprises:identifying an update operation as recorded in the roll-forward recovery log, wherein the update operation relates to the system table;identifying, using the partial image of the update operation as stored within the roll-forward recovery log, ...

COMMUNICATION APPARATUS AND COMMUNICATION METHOD

A non-transitory computer-readable recording medium having stored therein a program included in a communication apparatus that is configured to cause a computer to execute a procedure, the procedure includes when a first process related to first data is executed, arranging, in a pointer table included in a distribution ledger by which history related to distribution of data is managed in a data distribution system which includes the communication apparatus, pointer information which associates a source identifier which is an identifier of a history item of a process related to the first data performed before the first process is executed with a next identifier which is an identifier of a history item of the first process, and synchronizing content of the pointer table with content of a pointer table of another communication apparatus. 1. A non-transitory computer-readable recording medium having stored therein a program included in a communication apparatus that is configured to cause a computer to execute a procedure , the procedure comprising:when a first process related to first data is executed,arranging, in a pointer table included in a distribution ledger by which history related to distribution of data is managed in a data distribution system which includes the communication apparatus, pointer information which associates a source identifier which is an identifier of a history item of a process related to the first data performed before the first process is executed with a next identifier which is an identifier of a history item of the first process; andsynchronizing content of the pointer table with content of a pointer table of another communication apparatus.2. The non-transitory computer-readable recording medium according to claim 1 , wherein the pointer information includes a management Uniform Resource Locator (URL) of a history item of execution of the first process.3. The non-transitory computer-readable recording medium according to claim 1 , the ...

Systems and methods for data replication synchronization

Systems and methods are provided for storing a first data object comprising a first set of immutable components, the first data object being associated with a corresponding second data object stored by a remote replication system. A difference is determined between the first set of immutable components of the first data object and a second set of immutable components of the corresponding second data object. A subset of immutable components is identified from the first set of immutable components based on the difference. The subset of immutable components from the first set of immutable components is provided to the remote replication system over a communication network.

DATA REPLICATION SYSTEM

An improved data replication system with goals of Recovery Point Objective of zero, meaning a zero potential for loss of data, and a Recovery Time Objective in the seconds, meaning very short downtime after a failure. The system includes duplicates of a data set preferably on multiple drives in multiple computers. Change requests are stored within duplicates, and when a change request has been stored in enough duplicates, it may be applied to any duplicate after all prior change requests have been applied to that duplicate. The system applies changes to duplicates of a data set in the same order. The system further includes a replicator operable to implement accessing and changing steps to meet the goals. 1. A system for improving computer operations and data safety , the system enabling data replication with improved data loss prevention and time to recovery , the system comprising:a plurality of duplicates of a data set, the plurality of duplicates comprising a first duplicate and a second duplicate, wherein the first duplicate may be changed so that the data set on the first duplicate no longer exactly matches the data set on the second duplicate;a non-transient computer-readable memory device storing at least one portion of a duplicate in the plurality of duplicates; processing a retrieval request for data of the data set;', 'processing a request to change the data set;', 'assigning unique metadata to each request to change the data set, said unique metadata specifying a sequence to apply each request to change said data set;', 'storing in any duplicate of the data set a representation of each request to change that data set and the unique metadata assigned to each such request to change that data set;', 'storing in any duplicate of the data set an indication of which request to change that data set was applied last in said duplicate;', a specified minimum count of such duplicates storing the request to change the data set,', 'a specified required combination of ...

Cloud Queue Tombstone

Example techniques relate to cloud queue tombstones. An example implementation may involve a computing system receiving instructions to remove a particular media item from a playback queue. Based on the instructions, the computing system removes the particular media item from the playback queue and adds, to the playback queue, a tombstone indicator indicating that the particular media item has been removed from the playback queue. The computing system receives, from the media playback system, a request for a window of one or more media items from the playback queue. Based on the request, the computing system sends, to the media playback system, data representing a particular window of media items from the playback queue, wherein the particular window includes the tombstone indicator. The computing system detects a given tombstone removal trigger from among multiple tombstone removal triggers and based on the detecting, removes the tombstone indicator from the playback queue. 1. A method to be performed by a computing system , the method comprising:receiving, via a network interface, instructions to remove a particular media item from a playback queue, wherein one or more playback devices of a media playback system are configured to play back media items according to the playback queue;based on receiving the instructions to remove the particular media item from the playback queue, removing the particular media item from the playback queue;adding, to the playback queue, a tombstone indicator indicating that the particular media item has been removed from the playback queue;receiving, via the network interface from the media playback system, a request for a window of one or more media items from the playback queue;based on receiving the request for the window, sending, via the network interface to the media playback system, data representing a particular window of media items from the playback queue, wherein the particular window includes the tombstone indicator; ...

Efficient distributed lock manager

A system for managing a cluster of databases that process shared data, the system including a cluster of database nodes, each node including a memory, and a processor configured to, when it is determined that: (i) a lock on a resource is required by a particular node or process, and (ii) no other lock conflicts with this particular lock, send an asynchronous lock taken (ALT) message to a distributed lock manager (DLM); and without receiving a response or permission from the DLM, taking the lock for a resource for the particular node or process.

Recovering the Metadata of Data Backed Up in Cloud Object Storage

Techniques for recovering metadata associated with data backed up in cloud object storage are provided. In one set of embodiments, a computer system can create a snapshot of a data set, where the snapshot includes a plurality of data blocks of the data set that have been modified since the creation of a prior snapshot of the data set. The computer system can further upload the snapshot to a cloud object storage platform of a cloud infrastructure, where the snapshot is uploaded as a plurality of log segments conforming to an object format of the cloud object storage platform, and where each log segment includes one or more data blocks in the plurality of data blocks, and a set of metadata comprising, for each of the one or more data blocks, an identifier of the data set, an identifier of the snapshot, and a logical block address (LBA) of the data block. The computer system can then communicate the set of metadata to a server component running in a cloud compute and block storage platform of the cloud infrastructure.

ORCHESTRATION OF CRUD OPERATIONS FOR A HIERARCHICAL WEB SERVICE DATA MODEL IN A SPREADSHEET

Example embodiments facilitate orchestrating changes of data made (and/or actions specified for execution) in a client-side program with changes of corresponding data made in one or more server-side data objects. An example method includes structuring one or more worksheet relationships between one or more worksheets of a client-side spreadsheet in accordance with one or more data object relationships of one or more data objects of a server-side data structure; determining that one or more operations have been selected for performing on data of one or more of the worksheets; detecting user input operative to initiate performance of the one or more operations; and initiating implementation of the one or more operations (e.g., CRUD (Create, Read, Update, Delete) operations) in accordance with the one or more worksheet relationships and the one or more data object relationships (e.g., hierarchical relationships). A spreadsheet add-in may issue request messages to web services tasked with completing the one or more operations.

Data synchronization management between devices

A method, system, and computer readable medium for managing records. The method includes generating and storing, in response to detecting creation of a record, two values for the record, at least a first of the two values indicating a time of the creation of the record. The method also includes updating, in response to detecting a modification of the record, the first value while leaving the second of the two values constant. The method further includes determining, in response to identifying multiple versions of the record, which of the multiple versions of the record is a current version of the record based on the first value. Additionally, the method includes sending data associated with the current version of the record to at least one device not having the current version.

DEDUPLICATION OF DATA VIA ASSOCIATIVE SIMILARITY SEARCH

A deduplication system includes a similarity searcher, a difference calculator, and a storage manager. The similarity searcher searches for a similar fingerprint in a database storing a plurality of local sensitive fingerprints, resembling a new fingerprint of a new block. The difference calculator computes a difference block between the input block and a similar block associated with the found similar fingerprint, and the storage manager updates the database with the new fingerprint and stores the difference block, if not empty, in a store. A method for deduplication includes searching in a database, storing a plurality of local sensitive fingerprints, a similar fingerprint, resembling a new fingerprint of a new block, calculating a difference block between the input block and a similar block associated with the similar fingerprint, if found, updating the database with the new fingerprint and storing the difference block, if it is not empty, in a storage unit. 1. A deduplication system comprising:a similarity searcher to search, in a fingerprint database storing a plurality of fingerprints, for a similar fingerprint resembling a new fingerprint of an input block;a difference calculator to compute a difference block between said input block and a similar block associated with said similar fingerprint, if found; anda storage manager to update said fingerprint database with said new fingerprint and to store said difference block in a storage unit, if said difference block is not empty.2. The system of wherein said storage manager to store said input block if a similar fingerprint was not found.3. The system of wherein said storage manager to store fingerprints in columns of an associative memory device and wherein said similarity searcher performs said search inside said associative memory device.4. The system of also comprising:a fingerprint creator to create said new fingerprint using a locality-sensitive hashing (LSH) algorithm to create slightly different new ...

System and Method for Revising Record Keys to Coordinate Record Key Changes within at least Two Databases

A system and method to coordinate the changing of identifiers, i.e., primary keys and foreign keys, of new records generated by a first system by revision of these new records with alternate primary keys and foreign keys as generated and/or established by a second database system. The invention is an improvement over prior art transactional methods that maintain relationships among records when updating records of a first data base with replacement of primary and foreign keys provided by another database management system, and an improvement over prior art approaches to non-indexed schemas. Thus, the new method teaches how to batch update relationship fields in the first database with fewer database updates. The invention provides large savings for columnar databases, which are generally not indexed, and speeds up record key maintenance and harmonizing across two or more row-oriented databases. 1. A computer implemented method for management of key changes in relational references in an information technology system comprising a source database and a target database , the method comprising:a. Maintaining a plurality of source records in the source database, each record associating with a unique identifier;b. Generating a new source record within the source database;c. Associating a temporary unique identifier with the new source record, wherein the temporary unique identifier is applied as a unique identifier key of the new source record and distinguishes the new source record from the plurality of source records;d. Creating a new record in the target database as a copy of the source database record;e. Receiving a new target-assigned unique identifier, the new target-assigned unique identifier distinguishing the new source record from the plurality of source records and the temporary unique identifier;f. Writing the new target-assigned unique identifier and the temporary unique identifier into a key table as an associated key pair; andg. Revising the source database ...

NEAR-ZERO DOWNTIME RELOCATION OF A PLUGGABLE DATABASE ACROSS CONTAINER DATABASES

Embodiments minimize downtime involved in moving a PDB between CDBs by allowing read-write access to the PDB through most of the moving operation, and by transparently forwarding connection requests, for the PDB, from the source CDB to the destination CDB. The files of a source PDB are copied from a source CDB to a destination CDB, during which the source PDB may be in read-write mode. The source PDB is then closed to write operations so that changes to the source PDB cease. Another round of recovery is performed on the PDB clone, which applies all changes that have been performed on the source PDB during the copy operation and the PDB clone is opened for read and write operations. Forwarding information is registered with the source location, which information is used to automatically forward connection requests, received at the source location for the moved PDB, to the destination location. 1. A computer-executed method comprising:moving, to a destination container database, a particular pluggable database from a source container database;wherein the destination container database is distinct from the source container database; registering forwarding information with a database listener that is configured to receive connection requests to connect to the particular pluggable database at the source container database,', 'wherein the forwarding information includes connection information for the particular pluggable database at the destination container database;, 'in response to moving the particular pluggable database to the destination container databaseafter moving the particular pluggable database to the destination container database, the database listener receiving a particular connection request, from a particular user, to connect to the particular pluggable database at the source container database; andin response to receiving the particular connection request to connect to the particular pluggable database at the source container database and based, at ...

Robust Data Synchronization Solution Between Databases

Synchronizing data across databases comprising generating a synchronization event in response to a change in source data in a source database. The synchronization event is published as a message in a source database internal queue. The message is then published in a message broker. Sink data in a sink database is synchronized according to the message in the message broker to match the change in source data in the source database. 1. A computer-implement method of synchronizing data across databases , the method comprising:generating, by a number of processors, a synchronization event in response to a change in source data in a source database;publishing, by the number of processors, the synchronization event as a message in a source database internal queue;publishing, by the number of processors, the message in a message broker; andsynchronizing, by the number of processors, sink data in a sink database according to the message in the message broker to match the change in source data in the source database.2. The method of claim 1 , wherein data is synchronized between the source database and sink database in real time as data changes occur.3. The method of claim 1 , wherein data in the source database and sink database are correlated by Universal Unique Identifiers (UUIDs).4. The method of claim 1 , further comprising dividing claim 1 , by the number of processors claim 1 , source data in the source database into domains that match a data model of the sink database.5. The method of claim 1 , wherein the message is published in the source database internal queue in a first data format and published in the message broker in a second data format.6. The method of claim 1 , wherein a synchronization reactor consumes the message from the source database internal queue and pushes the message to the message broker.7. The method of claim 1 , wherein a domain reactor consumes the message from the message broker and updates sink data in the sink database.8. A system for ...

HANDLING METADATA OPERATIONS AND TIMESTAMP CHANGES DURING RESYNCHRONIZATION

Techniques are provided for resynchronizing a synchronous replication relationship. Asynchronous incremental transfers are performed to replicate data of a storage object to a replicated storage object. Incoming write requests, targeting the storage object, are logged into a dirty region log during a last asynchronous incremental transfer. Metadata operations, executed on the storage object, are logged into a metadata log during the last asynchronous incremental transfer. Sequence numbers are assigned to the metadata operations based upon an order of execution. The metadata operations are replicated to the replicated storage object for execution according to the sequence numbers, and the dirty regions are replicated to the replicated storage object in response to the metadata operations having been replicated to the replicated storage object. The storage object and replicated storage object are transitioned to a synchronous replication state where incoming operations are synchronously replicated to the replicated storage object. 1. A method comprising:asynchronously transferring data of a storage object of a first node to a replicated storage object at a second node;logging timestamp changes for the storage object and metadata operations executed upon the storage object according to an order of execution;replicating the timestamp changes and the metadata operations to the replicated storage object according to the order of execution; andreplicating modified data of the storage object to the replicated storage object upon the metadata operations being replicated.2. The method of claim 1 , wherein the metadata operations are logged into a metadata log claim 1 , and the method comprisingassigning sequence numbers to the metadata operations within the metadata log based upon the order of execution.3. The method of claim 2 , comprising:logging the timestamp changes into the metadata log.4. The method of claim 2 , wherein the metadata log is a queue within which the ...

Processing service requests for non-transactional databases

A method and apparatus for processing service requests for non-transactional databases are disclosed. In the method and apparatus, the execution of a first operation may be initiated based at least in part on receiving a service request from a user. The first operation may be executed synchronously with receipt of the request. Further, one or more execution workflows may be provisioned for asynchronous execution of one or more other operations.

Distributed ledger technology platform

A distributed ledger system is described. The system includes a provider to provide a plurality of infrastructure resources, a client to access a first set of the plurality of resources; and an operator platform to facilitate access to the first set of resources from the providers, including a processor to generate one or more blocks of transaction data associated with each resource in the first set of resources to update chain code and measure of usage the first set of resources measure, wherein the chain code is stored in a distributed ledger database shared between the operator platform and the client.

Terminal, storage medium, and database synchronization method thereof

A terminal, a storage medium, and a database synchronization method thereof are disclosed. The database synchronization method includes reading data from a first database and capturing a dynamic data log in the first database; joining data to a waiting queue read from the first database; determining a dislocation category of the data in the waiting queue, and rearranging the dislocation data in the waiting queue into a first category according to the dynamic data log; and synchronizing the rearranged data to a second database. According to the database synchronization method provided by the present invention caches the data read from the first database to the waiting queue, and dislocation data in the waiting queue is rearranged and then synchronized to the second database, so that a problem of poor data identity after database synchronization is avoided.

Database replication method and apparatus for distributed system

A database replication method and apparatus for a distributed system are provided and relate to the database field. The method includes: receiving by a coordination server, a timestamp of a multi-partition transaction newly added to a first partition of a secondary cluster; determining, by the coordination server, a target timestamp for the first partition based on the received timestamp of the newly added multi-partition transaction and a stored multi-partition transaction timestamp of each partition of the secondary cluster; and sending, by the coordination server, the target timestamp to the first partition, so that the first partition executes a replication log in the first partition based on the target timestamp. In this way, the corresponding partition can execute, without waiting, a multi-partition transaction that is present in all the partitions but has not been executed, thereby avoiding data inconsistency and increasing replication efficiency.

Rapid sync method for cloud file system and cloud file system using the same

Disclosed is a synchronization method of a cloud file system that enables a portable terminal to not consume excessive amount of battery power for data synchronization with the portable terminal such as a smart phone, enables a portable terminal and a stationary user terminal such as a personal computer to upload and download data in an appropriate time, and prevents time delay caused by the excessive traffic, during the data synchronization. To this end, the present disclosure provides a second terminal with data information associated with data when the data is uploaded from a first terminal, and requests and uploads a check message for determining whether the data is the latest data, terminates a data synchronization operation when it is determined that the data is not the latest data based on the check message of the second terminal, and transmits the data to the second terminal so as to request execution of a synchronization operation on the data when it is determined that the data is the latest data based on the check message of the second terminal.

ACL BASED OPEN TRANSACTIONS IN REPLICATION ENVIRONMENT

A method, computer program product, and computer system for handling open transactions in a data replication environment is provided. The method includes determining a database user that initiated an earliest open transaction in a transaction log. The method further includes, upon determining that the database user does not have access privileges for one or more tables of a target database corresponding to one or more tables of a source database associated with the earliest open transaction, skipping replication of the earliest open transaction to the target database, such that the data replication system no longer waits for the earliest open transaction to complete. The skipping includes discarding, from the transaction log, one or more entries related to the earliest open transaction, and identifying, in the transaction log, a next earliest open transaction. 1. A computer-implemented method comprising:receiving, by one or more processors, in a data replication system where data is being replicated from a source database to a target database, a transaction log identifying a plurality of open transactions for which the data replication system is waiting to complete before replicating to the target database;determining, by one or more computer processors, a database user that initiated a first open transaction, wherein the first open transaction is an earliest open transaction in the transaction log;upon determining that the database user does not have access privileges for one or more tables of the target database corresponding to one or more tables of the source database associated with the first open transaction, skipping, by one or more computer processors, replication of the first open transaction to the target database, such that the data replication system no longer waits for the first open transaction to complete, wherein the skipping includes discarding, from the transaction log, one or more entries related to the first open transaction, and identifying, in ...

Heterogeneous compression in replicated storage

Various embodiments for data management in a replicated storage environment, by a processor device, are provided. In one embodiment, a method comprises storing a plurality of data replicas under a plurality of heterogeneous compression algorithms, wherein one of the data replicas is optimized for a data operation.

METHODS AND SYSTEMS FOR GARBAGE DELETION IN A DOCUMENT DATABASE

Disclosed herein are exemplary systems and methods for garbage collection and/or deletion in a document database. The methods may include, for each change in a first change set, determining whether a first characteristic of the change is superseded by a second characteristic of a corresponding change in a second change set. The change of the first change set and the change of the second change set can pertain to a document attribute. The method may include determining whether the first change set is redundant with the second change set if each change of the first change set is superseded by a corresponding change of the second change set, and eliminating the first change set from the document database when the first change set is redundant with second change set. 1. A system for garbage deletion in a document database , the system comprising:at least one memory for storing computer-executable instructions; and [ comparing a first characteristic of the change to a second characteristic of a corresponding change in a second change set, wherein the change of the first change set and the change of the second change set pertain to an attribute of a document in the document database;', 'determining whether the change of the first change set is superseded by the change of the second change set based on the comparison of the first characteristic to the second characteristic;, 'for each change in a first change set, 'determining whether the first change set is redundant with the second change set if each change of the first change set is superseded by a corresponding change of the second change set; and', 'eliminating the first change set from the document database when the first change set is redundant with second change set., 'at least one processor for executing the instructions stored on the memory, wherein execution of the instructions programs the at least one processor to perform operations comprising2. The system of claim 1 , wherein the first characteristic is a ...

REMOTE COPY SYSTEM

A first storage system of a remote copy system includes a data volume, a journal volume, and a storage controller. When an operation is received from a first server system, based on the operation, the storage controller executes any of a work of storing the update content by the operation in the journal volume or a work of executing the operation when the operation is received. 1. A remote copy system comprising:a primary site including a first server system and a first storage system connected to the first server system; anda secondary site including a second server system and a second storage system connected to the second server system, whereinthe first server system and the second server system are clustered,a virtual storage system is configured by the first storage system and the second storage system,an asynchronous remote copy is performed with the first storage system as a copy source and the second storage system as a copy destination,the first and second storage systems are configured to detect a failure of the second or first storage system that is a destination of the asynchronous remote copy,the first storage system includes a data volume, a journal volume, and a storage controller, andwhen an operation is received from the first server system, based on the operation, the storage controller executes any of a work of storing the update content by the operation in the journal volume or a work of executing the operation after the operation is received.2. The remote copy system according to claim 1 , whereinthe storage controller reflects the update content stored in the journal volume on the second server system after a failover occurs at the primary site.3. The remote copy system according to claim 2 , whereinthe storage controller determines which work is executed based on a type of the operation.4. The remote copy system according to claim 3 , whereinthe storage controller selects a method of reflecting the update content stored in the journal volume ...

Asynchronous document ingestion and enrichment system

Provided are asynchronous data ingestion and enrichment systems and methods. The systems comprise a plurality of components (e.g., ingestion components, enrichment components, and/or publishing components). Instead of passing data from one component to another, the data is sent to a messaging queue that formats and hold the data until the subsequent component is ready to receive it. Additionally, each component comprises a central microservice and a plurality of instances, the central microservice configured to communicate with each instance of the plurality of instances.

Making a user's data, settings, and licensed content available in the cloud

A cloud-based computer system changes the modern paradigm from being device-centric to being person-centric. The system makes all user data, software settings, device settings, and licensed content for a user available in the cloud. The system includes a conversion mechanism that can convert information intended for one device type to a different device type. Thus, a user changing smart phone platforms can convert their current smart phone settings to equivalent settings on the new phone platform, and their new phone can then be configured using the user's converted settings stored in the cloud. By storing all the user's relevant information in the cloud, this information may be accessed anywhere and may be used to configure a large number of different devices according to the user's settings.

MAINTAINING CONSISTENCY OF ASYNCHRONOUS REPLICATION

Maintaining consistency for asynchronous replication of LUNs of a local storage system to corresponding LUNs of a remote storage system includes copying application data written to LUNs to cache slots, copying metadata of LUNs to cache slots, associating a first sequence number to all metadata modifications and all application data writes that are begun after a first time and before a second time, associating a second sequence number, different from the first sequence number, to all metadata modifications and all application data writes that are begun after the second time, and, after completion of all metadata modifications and all application data writes associated with the first sequence number, initiating transfer to the remote storage system of all metadata modifications and all application data writes associated with the first sequence number. Each cache slot may include a field that indicates a corresponding sequence number. 1. A method of maintaining consistency for asynchronous replication of LUNs of a local storage system to corresponding LUNs of a remote storage system , comprising:copying application data written to LUNs of the local storage system to corresponding cache slots;copying metadata of LUNs of the local storage system to corresponding cache slots of the local storage system in response to modifying the metadata of the LUNs;associating a first sequence number to all metadata modifications and all application data writes for any of the LUNs of the local storage system that are begun after a first time and before a second time;associating a second sequence number, different from the first sequence number, to all metadata modifications and all application data writes for any of the LUNs of the local storage system that are begun after the second time; andafter completion of all metadata modifications and all application data writes associated with the first sequence number, initiating transfer to the remote storage system of all metadata ...

PARTIAL TABLE AND MULTISOURCE SYNCHRONIZATION FOR DATABASES

A database system provides automatic synchronization from one or more databases to a table. The synchronized portion of each source database table is periodically imported into a corresponding portion of a destination database table. For each synchronized field in the table with data from multiple source databases, one of the source databases is set as a primary source, which determines the data type and field configuration of the field. Data from secondary sources are cast to the data type established by the primary source and represented using the primary source's field configuration. The source database table may also include data that is not synchronized with the destination database table. Similarly, the destination database table may be enriched with data that is not included in the source database table. 1. A computer-implemented method for data synchronization , comprising:receiving, by a server, a request to synchronize data from a first base to a second base, the first base having data arranged in rows and columns, wherein the request identifies a set of data in the first base to synchronize to the second base;configuring, by the server, a periodic synchronization of the set of data from the first base to the second base;receiving, by the server, a change to the set of data in the first base; andpropagating, by the server as part of a next instance of the periodic synchronization, the change to the set of data in the first base to the second base.2. The computer-implemented method of claim 1 , further comprising:receiving, by the server, a request to synchronize a second set of data from the second base to a third base, wherein the second set of data comprises at least a portion of the set of data; andconfiguring, by the server, a second periodic synchronization of the second set of data from the second base to the third base.3. The computer-implemented method of claim 1 , wherein a period of the periodic synchronization is a length of time ranging from ...

Parallel processing of disjoint change streams into a single stream

Technology for tracking changes to a database and providing the changes to a subscribing computing device. In an illustrative implementation, a method may involve subscribing a computing device to changes of a portion of a multi-tenant database and accessing a plurality of change events stored in a log of the multi-tenant database in parallel. The method may further involve storing the plurality of change events in a persistent data store separate from the multi-tenant database and processing the plurality of change events to create an ordered set of change events. The ordered set of change events may then be provided to the computing device that subscribed to the changes.

Stateless datastore-independent transactions

At a client-side component of a storage group, a read descriptor generated in response to a read request directed to a first data store is received. The read descriptor includes a state transition indicator corresponding to a write that has been applied at the first data store. A write descriptor indicative of a write that depends on a result of the read request is generated at the client-side component. The read descriptor and the write descriptor are included in a commit request for a candidate transaction at the client-side component, and transmitted to a transaction manager.

JOURNALING DATA RECEIVED IN A CLOUD-BASED DISTRIBUTED COMPUTING ENVIRONMENT

The disclosed embodiments disclose techniques for journaling data received in a cloud-based distributed computing environment (CBDCE). Multiple services simultaneously execute on the CBDCE compute nodes, with each service comprising multiple service instances that simultaneously execute on multiple, distinct compute nodes of the CBDCE. The CBDCE includes a distributed database that enables coordination between the service instances of services that execute in the CBDCE; this distributed database also includes multiple distributed database instances that simultaneously executing on multiple different CBDCE compute nodes. During operation, a service instance executing on one of these compute nodes receives a client request. The service instance submits this client request to a distributed database instance and, in parallel, also submits the client request and its associated user data to a distributed journaling service. 1. A computer-implemented method for journaling data received in a cloud-based distributed computing environment (CBDCE) , the method comprising:receiving at a service instance a client request that includes a set of user data, wherein the CBDCE comprises multiple geographically-distributed compute nodes, wherein multiple services simultaneously execute on the CBDCE compute nodes, wherein each service comprises multiple service instances that simultaneously execute on multiple distinct compute nodes of the CBDCE, wherein the service instance is an instance of a specific service and is executing on a specific CBDCE compute node, wherein the CBDCE includes a distributed database that enables coordination between the service instances of one or more services that execute in the CBDCE, wherein the distributed database comprises a set of multiple distributed database instances simultaneously executing on multiple different CBDCE compute nodes;submitting the client request to a distributed database instance executing on a second CBDCE compute node that is ...

PROVIDING CONSISTENT DATABASE RECOVERY AFTER DATABASE FAILURE FOR DISTRIBUTED DATABASES WITH NON-DURABLE STORAGE LEVERAGING BACKGROUND SYNCHRONIZATION POINT

An approach for consistent database recovery for distributed database systems uses “synchronization points”. A synchronization point is a global timestamp for which across all nodes of a distributed database system, the nodes have stored change records for any transaction occurring at and before the synchronization point in persistent logs. Each node may employ asynchronous flushing. However, on a periodic basis, each node coordinates to establish a synchronization point, which may entail ensuring change records for transactions that committed at or before the synchronization point are stored in persistent logs. In effect, a synchronization point represents that any transaction committed at or before the synchronization point has been durably committed. 1. A method comprising:generating a plurality of change records corresponding to a plurality of database transactions executed by a multi-node database system against a database, wherein each node of said multi-node database system generates a respective set of change records of said plurality of change records;said multi-node database system generating a plurality of synchronization points that each correspond to a respective global timestamp of said multi-node database system, wherein for each synchronization point of said plurality of synchronization points, each node of said multi-node database system has persistently stored all change records of the respective set of change records of said each node for any database transaction of said plurality of database transactions that has committed before or no later than the respective global timestamp of said each synchronization point;for a particular synchronization point that corresponds to a particular global timestamp of said multi-node database system, recovering the database by at least, for each node of said multi-node database system, applying at least a respective subset of change records of the respective set of change records of said each node to roll ...

COMPUTER STORAGE DEDUPLICATION

Decentralized deduplication operations in a computer system employ a hash index that is a variant of a B+ tree to support both efficient sequential updates as well as efficient random updates. Sequential update is selected when deduplication is infrequently performed, such as on the order of days, and random update is selected when deduplication is performed more frequently, such as on the order of seconds. More frequent deduplication may be beneficial during periods when large amounts of temporary duplicate data are created, and the system may not have enough storage space to accommodate the temporary spike in demand. 1. A method for performing a deduplication operation in a computer system having multiple host computer systems connected to a common storage system , the method comprising:at each host computer system, tracking write operations to the common storage system; and one or more jump indexes including a top-level jump index, each jump index being divided into pages and having entries that include (a) a pointer to a first page in a next lower level jump index and (b) a hash value that is equal to a first entry in the first page of the next lower level jump index, each jump index having its entries maintained in sorted order according to the hash value in the jump index entry; and', 'a hash index that is divided into pages, each entry in the hash index containing a hash value of a block stored in the common storage system and a file pointer associated with the block, the hash index having its entries maintained in sorted order according to the hash value in the entry, and one or more of the jump indexes each having entries that include a pointer to each one of the pages of the hash index and a first hash value in the pointed-to page., 'performing the deduplication operation on storage blocks associated with the write operations by using a data structure to find duplicate blocks, the data structure including2. The method of claim 1 , wherein the deduplication ...

OFFLOADING DATA TO A COLD STORAGE DATABASE

Methods, systems, and devices supporting data archiving are described. A device (e.g., an application server) may identify a set of archiving jobs for a first database (e.g., a relational database), where each archiving job indicates a set of data records for archiving. The device may receive a data record of the set of data records from the first database, where the data record is filtered by an abstraction layer (e.g., removing some information from the data record such that the filtered data record corresponds to a user's view of the data record). For example, the filtered data record may include a subset of the total information for the stored data record. The device may transform this received, filtered data record from a first format to a second format supported by a second database. The device may then send the transformed data record to the second database for storage. 1. A method for data archiving from a first database to a second database , comprising:receiving, from the first database, a data queue indicating a set of archiving jobs;identifying a set of data records in the first database based at least in part on a first batch of the set of archiving jobs, wherein the set of data records is stored in a first format supported by the first database and a stored data record of the set of data records comprises a set of information;filtering, at a first abstraction layer and based at least in part on identifying the set of data records in the first database, the stored data record of the set of data records to obtain a filtered data record, wherein the filtered data record comprises a subset of the set of information for the stored data record;receiving, via the first abstraction layer, the filtered data record of the set of data records based at least in part on the first batch of the set of archiving jobs;transforming the received data record from the first format to a second format different from the first format and supported by the second database; ...