Patent application number | Description | Published |
20090234821 | Systems and Methods for Efficient Data Searching, Storage and Reduction - Systems and methods enabling search of a repository for the location of data that is similar to input data, using a defined measure of similarity, in a time that is independent of the size of the repository and linear in a size of the input data, and a space that is proportional to a small fraction of the size of the repository. The similar data segments thus located are further analyzed to determine their common (identical) data sections, regardless of the order and position of the common data sections in the repository and input, and in a time that is linear in the segment size and in constant space. | 09-17-2009 |
20090234855 | Systems and Methods for Efficient Data Searching, Storage and Reduction - Systems and methods enabling search of a repository for the location of data that is similar to input data, using a defined measure of similarity, in a time that is independent of the size of the repository and linear in a size of the input data, and a space that is proportional to a small fraction of the size of the repository. The similar data segments thus located are further analyzed to determine their common (identical) data sections, regardless of the order and position of the common data sections in the repository and input, and in a time that is linear in the segment size and in constant space. | 09-17-2009 |
20120117384 | METHOD AND SYSTEM FOR DELETING DATA - Methods, computer systems, and computer program products for deleting data in a computing environment are provided. A computer system having at least first and second documents, a plurality of decryption keys, and a plurality of data segments stored therein is provided. Each of the plurality of data segments is decryptable by a selected one of the decryption keys. The decryption keys include a first set of decryption keys associated with the first document and not associated with the second document, a second set of decryption keys associated with the second document and not associated with the first document, and a third set of decryption keys associated with the first document and the second document. The first document is deleted, and in response, the first set of decryption keys is rendered unusable, and the second set of decryption keys and the third set of decryption keys are not rendered unusable. | 05-10-2012 |
20120131082 | COMPUTATION OF A REMAINDER BY DIVISION USING PSEUDO-REMAINDERS - Methods, computer systems, and computer program products for calculating a remainder by division of a sequence of bytes interpreted as a first number by a second number is provided. A pseudo-remainder by division associated with a first subsequence of the sequence of bytes is calculated. A property of this pseudo-remainder is that the first subsequence of the sequence of bytes, interpreted as a third number, and the pseudo-remainder by division have the same remainder by division when divided by the second number. A second subsequence of the sequence of bytes interpreted as the first number is appended to the pseudo-remainder, interpreted as a sequence of bytes, so as to create a sequence of bytes interpreted as a fourth number. The first number and the fourth number have the same remainder by division when divided by the second number. | 05-24-2012 |
20120143835 | EFFICIENT CONSTRUCTION OF SYNTHETIC BACKUPS WITHIN DEDUPLICATION STORAGE SYSTEM - Various embodiments are provided for facilitating construction of a synthetic backup in a deduplication storage system. In one embodiment, a deduplication storage system enables new input data to be deduplicated with data of synthetic backups already constructed, and for this purpose efficiently calculates deduplication digests for synthetic backups being constructed, based on already existing digests of data referenced by the synthetic backups. For each input data segment of the plurality of input data segments of a synthetic backup being constructed, a plurality of deduplication digests of stored data segments, referenced by the input data segment, is retrieved from an index. Each input data segment is partitioned into each of a plurality of fixed-sized data sub-segments. A calculation is performed producing a deduplication digest for a data sub-segment, where the calculation is based on the retrieved deduplication digests of the plurality of stored data sub-segments referenced by the input data sub-segment. | 06-07-2012 |
20120158660 | METHOD AND SYSTEM FOR DEDUPLICATING DATA - Methods, computer systems, and computer program products for deduplicating data in a computing environment are provided. A sequence of data is received. The sequence of data is formatted for back-up such that an order of the sequence of data is different than the order of an input sequence of the data. The sequence of data is stored in the same order as the input sequence of the data. | 06-21-2012 |
20120158671 | METHOD AND SYSTEM FOR PROCESSING DATA - Methods, computer systems, and computer program products for processing data a computing environment are provided. The computer environment for data deduplication storage receives a plurality of write operations for deduplication storage of the data. The data is buffered in a plurality of buffers with overflow temporarily stored to a memory hierarchy when the data received for deduplication storage is sequential or non sequential. The data is accumulated and updated in the plurality of buffers per a data structure, the data structure serving as a fragment map between the plurality of buffers and a plurality of user file locations. The data is restructured in the plurality of buffers to form a complete sequence of a required sequence size. The data is provided as at least one stream to a stream-based deduplication algorithm for processing and storage. | 06-21-2012 |
20120158812 | PARALLEL COMPUTATION OF A REMAINDER BY DIVISION OF A SEQUENCE OF BYTES - Methods, computer systems, and computer program products for calculating a remainder by division of a sequence of bytes interpreted as a first number by a second number are provided. A first remainder by division associated with a first subset of the sequence of bytes is calculated with a first processor. A second remainder by division associated with a second subset of the sequence of bytes is calculated with a second processor. The calculating of the second remainder by division may occur at least partially during the calculating of the first remainder by division. A third remainder by division is calculated based on the calculating of the first remainder by division and the calculating of the second remainder by division. | 06-21-2012 |
20120239625 | EFFICIENT CONSTRUCTION OF SYNTHETIC BACKUPS WITHIN DEDUPLICATION STORAGE SYSTEM - A deduplication storage system enables new input data to be deduplicated with data of synthetic backups already constructed, and for this purpose efficiently calculates deduplication digests for synthetic backups being constructed, based on already existing digests of data referenced by the synthetic backups. For each input data segment of the plurality of input data segments of a synthetic backup being constructed, a plurality of deduplication digests of stored data segments, referenced by the input data segment, is retrieved from an index. Each input data segment is partitioned into each of a plurality of fixed-sized data sub-segments. A calculation is performed producing a deduplication digest for a data sub-segment, where the calculation is based on the retrieved deduplication digests of the plurality of stored data sub-segments referenced by the input data sub-segment. | 09-20-2012 |
20120243688 | METHOD AND SYSTEM FOR DELETING DATA - A computer system having at least first and second documents, a plurality of decryption keys, and a plurality of data segments stored therein is provided. Each of the plurality of data segments is decryptable by a selected one of the decryption keys. The decryption keys include a first set of decryption keys associated with the first document and not associated with the second document, a second set of decryption keys associated with the second document and not associated with the first document, and a third set of decryption keys associated with the first document and the second document. The first document is deleted, and in response, the first set of decryption keys is rendered unusable, and the second set of decryption keys and the third set of decryption keys are not rendered unusable. | 09-27-2012 |
20120271798 | METHOD AND SYSTEM FOR DEDUPLICATING DATA - For deduplicating data in a computing environment, a sequence of data is received. The sequence of data is formatted for back-up such that an order of the sequence of data is different than the order of an input sequence of the data. The sequence of data is stored in the same order as the input sequence of the data. | 10-25-2012 |
20120271873 | PARALLEL COMPUTATION OF A REMAINDER BY DIVISION OF A SEQUENCE OF BYTES - A remainder by division of a sequence of bytes interpreted as a first number by a second number is calculated. A first remainder by division associated with a first subset of the sequence of bytes is calculated with a first processor. A second remainder by division associated with a second subset of the sequence of bytes is calculated with a second processor. The calculating of the second remainder by division may occur at least partially during the calculating of the first remainder by division. A third remainder by division is calculated based on the calculating of the first remainder by division and the calculating of the second remainder by division. | 10-25-2012 |
20130073528 | SCALABLE DEDUPLICATION SYSTEM WITH SMALL BLOCKS - For scalable data deduplication working with small data chunks in a computing environment, for each of the small data chunks, a signature is generated based on a combination of a representation of characters that appear in the small data chunks with a representation of frequencies of the small data chunks. The signature is used to help in selecting the data to be deduplicated. | 03-21-2013 |
20130073529 | SCALABLE DEDUPLICATION SYSTEM WITH SMALL BLOCKS - For scalable data deduplication working with small data chunks in a computing environment, for each of the small data chunks, a signature is generated based on a combination of a representation of characters that appear in the small data chunks with a representation of frequencies of the small data chunks. The signature is used to help in selecting the data to be deduplicated. | 03-21-2013 |
20130179759 | INCREMENTAL MODIFICATION OF AN ERROR DETECTION CODE BACKGROUND OF THE INVENTION - Exemplary method, system, and computer program product embodiments for an incremental modification of an error detection code operation are provided. In one embodiment, by way of example only, for a data block requiring a first error detection code (EDC) value to be calculated and verified and is undergoing modification for at least one randomly positioned sub-blocks that becomes available and modified in independent time intervals, a second EDC value is calculated for each of the randomly positioned sub-blocks. An incremental effect of the second EDC value is applied for calculating the first EDC value and for recalculating the first EDC value upon replacing at least one of the randomly positioned sub-blocks. The resource consumption is proportional to the size of at least one of the randomly positioned sub-blocks that are added and modified. Additional system and computer program product embodiments are disclosed and provide related advantages. | 07-11-2013 |
20130232116 | CALCULATING DEDUPLICATION DIGESTS FOR A SYNTHETIC BACKUP BY A DEDUPLICATION STORAGE SYSTEM - Input backup data is deduplicated with data of a synthetic backup previously constructed by a deduplication storage. A synthetic backup is constructed by processing metadata instructions provided by a backup application. Deduplication digests are calculated based on the data of the synthetic backup and the deduplication digests are stored in a digests index. When new backup data is processed, deduplication digests of the new data are calculated and searched in the digests index. Matching digests of previously constructed synthetic backups are located in the digests index. Each of the located matching digest references stored data are included in the synthetic backup, and the stored data is similar to the input backup data. Data matches are found in the input data and data in the synthetic backup. | 09-05-2013 |
20130232117 | CREATION OF SYNTHETIC BACKUPS WITHIN DEDUPLICATION STORAGE SYSTEM BY A BACKUP APPLICATION - A deduplication storage system and a backup application create a synthetic backup. Metadata instructions are provided to the deduplication storage system. Each of the metadata instructions specifies the data segment of an originating backup and a designated location of the data segment in the synthetic backup. A set of metadata instructions is transformed into a transformed set of metadata instructions. | 09-05-2013 |
20130232119 | CREATION OF SYNTHETIC BACKUPS WITHIN DEDUPLICATION STORAGE SYSTEM - A deduplication storage system and a backup application create a synthetic backup. Metadata instructions are provided to the deduplication storage system. Each of the metadata instructions specifies the data segment of an originating backup and a designated location of the data segment in the synthetic backup. Each of the metadata instructions are processed by locating those data sub-segments in the deduplication storage system specified by the data segment in each of the metadata instructions, and creating metadata references to each of the data sub-segments and adding the metadata references to metadata of the synthetic backup being created. | 09-05-2013 |
20130232120 | DEDUPLICATING INPUT BACKUP DATA WITH DATA OF A SYNTHETIC BACKUP PREVIOUSLY CONSTRUCTED BY A DEDUPLICATION STORAGE SYSTEM - Input backup data is deduplicated with data of a synthetic backup previously constructed by a deduplication storage system. A synthetic backup is constructed by processing metadata instructions provided by a backup application. Deduplication digests are calculated based on the data of the synthetic backup and the deduplication digests are stored in a digests index. When new backup data is processed, deduplication digests of the new data are calculated and searched in the digests index. Matching digests of previously constructed synthetic backups are located in the digests index. Each of the located matching digest references stored data are included in the synthetic backup, and the stored data is similar to the input backup data. Data matches are found in the input backup data and data in the synthetic backup. | 09-05-2013 |
20130290278 | SCALABLE DEDUPLICATION SYSTEM WITH SMALL BLOCKS - Exemplary method, system, and computer program product embodiments for scalable data deduplication working with small data chunk in a computing environment are provided. In one embodiment, by way of example only, for each of the small data chunk, a signature is generated based on a combination of a representation of characters that appear in the small data chunk with a representation of frequencies of the small data chunk. A signature is generated based on a combination of a representation of characters that appear. The signature is used to help in selecting the data to be deduplicated. Additional system and computer program product embodiments are disclosed and provide related advantages. | 10-31-2013 |
20130290279 | SCALABLE DEDUPLICATION SYSTEM WITH SMALL BLOCKS - Exemplary method, system, and computer program product embodiments for scalable data deduplication working with small data chunk in a computing environment are provided. In one embodiment, by way of example only, for each of the small data chunk, a signature is generated based on a combination of a representation of characters that appear in the small data chunk with a representation of frequencies of the small data chunk. A signature is generated based on a combination of a representation of characters that appear. The signature is used to help in selecting the data to be deduplicated. Additional system and computer program product embodiments are disclosed and provide related advantages. | 10-31-2013 |
20130339316 | PACKING DEDUPLICATED DATA INTO FINITE-SIZED CONTAINERS - Deduplicated data is packed into finite-sized containers. A similarity score is calculated between files that are similarly of the deduplicated data. The similarity score is used for grouping the similarly compared files of the deduplicated data into subsets for destaging each of the subsets from a deduplication system to one a finite-sized container. | 12-19-2013 |
20140012822 | SUB-BLOCK PARTITIONING FOR HASH-BASED DEDUPLICATION - Sub-block partitioning for hash-based deduplication is performed by defining a minimal size and maximum size of the sub-block. For each boundary start position of the sub-block, starting a search, after the minimal size of the sub-block, for a boundary position of a subsequent sub-block by using multiple search criteria to test hash values that are calculated during the search. If one of the multiple search criteria is satisfied by one of the hash values, declaring the position of the hash value as a boundary end position of the sub-block. If the maximum size of the sub-block is reached prior to satisfying one of the multiple search criteria, declaring a position of an alternative one of the hash values that is selected based upon another one of the multiple search criteria as the boundary end position of the sub-block. | 01-09-2014 |
20140089269 | EFFICIENT FILE RECLAMATION IN DEDUPLICATING VIRTUAL MEDIA - Expired files in the deduplicating virtual media are selectively erased using a backup application for notifying a backup repository of which expired files are no longer required. The space of the expired files is reclaimed for reuse. Virtual space of the expired files is reserved for allowing the backup application to seek past the reclaimed space to subsequent data in the deduplicating virtual media. | 03-27-2014 |
20140089275 | EFFICIENT FILE RECLAMATION IN DEDUPLICATING VIRTUAL MEDIA - Expired files in the deduplicating virtual media are selectively erased using a backup application for notifying a backup repository of which expired files are no longer required. The space of the expired files is reclaimed for reuse. Virtual space of the expired files is reserved for allowing the backup application to seek past the reclaimed space to subsequent data in the deduplicating virtual media. | 03-27-2014 |
20140101114 | METHOD AND SYSTEM FOR PROCESSING DATA - Methods, computer systems, and computer program products for processing data a computing environment are provided. The computer environment for data deduplication storage receives a plurality of write operations for deduplication storage of the data. The data is buffered in a plurality of buffers with overflow temporarily stored to a memory hierarchy when the data received for deduplication storage is sequential or non sequential. The data is accumulated and updated in the plurality of buffers per a data structure, the data structure serving as a fragment map between the plurality of buffers and a plurality of user file locations. The data is restructured in the plurality of buffers to form a complete sequence of a required sequence size. The data is provided as at least one stream to a stream-based deduplication algorithm for processing and storage. | 04-10-2014 |
20140188818 | OPTIMIZING A PARTITION IN DATA DEDUPLICATION - For optimizing a partition of a data block into matching and non-matching segments in data deduplication using a processor device in a computing environment, an optimal calculation operation is applied in polynomial time to the matching segments for selecting a globally optimal subset of a set of matching segments according to overhead considerations for minimizing an overall size of a deduplicated file by determining a trade off between a time complexity and a space complexity. | 07-03-2014 |
20140188828 | CONTROLLING SEGMENT SIZE DISTRIBUTION IN HASH-BASED DEDUPLICATION - Segment sizes are controlled by setting the size of a segment boundary in a hash-based deduplication system. A subsequence of size K of a sequence of characters S is set. An increasing sequence of n probabilities and a corresponding sequence of n decreasingly restrictive logical tests are chosen to be applied on the sequence of characters S. Segment boundaries are set by using the sequence of the decreasingly restrictive logical tests by deciding to declare a segment boundary at a current position if one of the sequence of the decreasingly restrictive logical tests, with a corresponding probability of the sequence of n probabilities, returns a true value when applied on the sequence of characters S. | 07-03-2014 |
20140195493 | PACKING DEDUPLICATED DATA IN A SELF-CONTAINED DEDUPLICATED REPOSITORY - Deduplicated data is packed in a self-contained deduplicated repository having unique data blocks with each being referenced by a globally unique identifier (GUID). The self-contained deduplicated repository has information regarding both deduplicated data files and the unique data blocks of each of the deduplicated data files and a master GUID list containing a location of each of the unique data blocks. | 07-10-2014 |
20140195495 | PACKING DEDUPLICATED DATA IN A SELF-CONTAINED DEDUPLICATED REPOSITORY - Deduplicated data is packed in a self-contained deduplicated repository having unique data blocks with each being referenced by a globally unique identifier (GUID). The self-contained deduplicated repository has information regarding both deduplicated data files and the unique data blocks of each of the deduplicated data files and a master GUID list containing a location of each of the unique data blocks. | 07-10-2014 |
20140201480 | Data Recovery Scheme Based on Data Backup Status - Machines, systems and methods for increasing data resiliency in a computing system, the method comprising distinguishing between first data and second data stored in one or more data storage mediums, wherein the first data is more vulnerable than the second data for the purpose of recovering lost data; and recovering the first data before recovering the second data. Increasing redundancy protection for the first data to increase chances for data recovery by way of data reconstruction; and decreasing redundancy protection for the first data, after the first data has been backed up at least once. | 07-17-2014 |
20140279951 | DIGEST RETRIEVAL BASED ON SIMILARITY SEARCH IN DATA DEDUPLICATION - For digest retrieval based on similarity search in deduplication processing in a data deduplication system using a processor device in a computing environment, input data is partitioned into fixed sized data chunks. Similarity elements and digest block boundaries and digest values are calculated for each of the fixed sized data chunks. Matching similarity elements are searched for in a search structure containing the similarity elements for each of the fixed sized data chunks in a repository of data. Positions of similar data are located in the repository. The positions of the similar data are used to locate and load into the memory stored digest values and corresponding stored digest block boundaries of the similar data in the repository. The digest values and the corresponding digest block boundaries of the input data are matched with the stored digest values and the corresponding stored digest block boundaries to find data matches. | 09-18-2014 |
20140279952 | EFFICIENT CALCULATION OF SIMILARITY SEARCH VALUES AND DIGEST BLOCK BOUNDARIES FOR DATA DEDUPLICATION - For efficient calculation of both similarity search values and boundaries of digest blocks in data deduplication, input data is partitioned into chunks, and for each chunk a set of rolling hash values is calculated. A single linear scan of the rolling hash values is used to produce both similarity search values and boundaries of the digest blocks of the chunk. | 09-18-2014 |
20150019502 | READ AHEAD OF DIGESTS IN SIMILARITY BASED DATA DEDUPLICATON - For read ahead of digests in similarity based data deduplication in a data deduplication system using a processor device in a computing environment, input data is partitioned into data chunks and digest values are calculated for each of the data chunks. The positions and sizes of similar data intervals in a repository of data are found for each of the data chunks. The positions and the sizes of read ahead intervals are calculated based on the similar data intervals. The read ahead digests of the read ahead intervals are located and loaded into memory in a background read ahead process. | 01-15-2015 |
20150019503 | DIGEST BLOCK SEGMENTATION BASED ON REFERENCE SEGMENTATION IN A DATA DEDUPLICATION SYSTEM - For producing digest block segmentations based on reference segmentations in a data deduplication system using a processor device in a computing environment, digests are calculated for an input data chunk. Data matches and data mismatches are produced based on matching input digests with reference digests. Secondary digest block segmentations are obtained from similar reference intervals for each of the data mismatches and applied to the input data. | 01-15-2015 |
20150058297 | ADDING COOPERATIVE FILE COLORING PROTOCOLS IN A DATA DEDUPLICATION SYSTEM - For adding cooperative file coloring protocols in a data deduplication system using a processor device in a computing environment, a preferred character is represented for file coloring in a file using a code selected from a multiplicity of codes that represent a variety of contexts. The original meaning of the preferred character is retained when representing the preferred character for the file coloring by the code selected from the multiplicity of codes. | 02-26-2015 |
20150088843 | OPTIMIZING A PARTITION IN DATA DEDUPLICATION - For optimizing a partition of a data block into matching and non-matching segments in data deduplication using a processor device in a computing environment, a sequence of matching segments is split into sub-parts for obtaining a globally optimal subset, to which an optimal calculation is applied. The solutions of optimal calculations for the entire range of the sequence are combined, and a globally optimal subset is built by means of a first two-dimensional table represented by a matrix C[i, j], and storing a representation of the globally optimal subset in a second two-dimensional table represented by a matrix PS[i, j] that holds, at entry [i, j] of the matrix, the globally optimal subset for a plurality of parameters in form of a bit-string of length j−i+1, wherein i and j are indices of bit positions corresponding to segments. | 03-26-2015 |