Patent application number | Description | Published |
20120233117 | Higher Efficiency Storage Replication Using Compression - An improved scalable object storage system includes methods and systems allowing multiple clusters to work together. In one embodiment, there is a multi-cluster synchronization system between two or more clusters. The multi-cluster synchronization system uses variable compression to optimize the transfer of information between the clusters. Compression is used not only to minimize the total number of bytes sent between the two clusters, but to dynamically vary the size of the objects sent across the wire to optimize for higher throughput after considering packet loss, TCP windows, and block sizes. This includes both the packaging of multiple small files together into one larger compressed file, saving on TCP and header overhead, but also the chunking of large files into multiple smaller files that are less likely to have difficulties due to intermittent network congestion or errors. | 09-13-2012 |
20120233118 | Multi-Cluster Container Replication - An improved scalable object storage system allows multiple clusters to work together. In one embodiment, a multi-cluster synchronization system synchronizes data between two or more clusters. Each cluster has a cluster-internal network, with object storage services and container services. The container services track and replicate metadata associated with the object storage service. An intercluster network connects the two clusters and performs a one-way synchronization of the objects and metadata associated with a particular container. In a further embodiment, multiple synchronization relationships can be set up, either in a cycle (with two or more participants), in a line, or in a tree. For example, the multi-cluster replication could be used to transparently synchronize objects in a CDN network. | 09-13-2012 |
20120233119 | Openstack database replication - Several different embodiments of a massively scalable object storage system are described. The object storage system is particularly useful for storage in a cloud computing installation whereby shared servers provide resources, software, and data to computers and other devices on demand. In several embodiments, the object storage system includes a ring implementation used to associate object storage commands with particular physical servers such that certain guarantees of consistency, availability, and performance can be met. In other embodiments, the object storage system includes a synchronization protocol used to order operations across a distributed system. In a third set of embodiments, the object storage system includes a metadata management system. In a fourth set of embodiments, the object storage system uses a structured information synchronization system. Features from each set of embodiments can be used to improve the performance and scalability of a cloud computing object storage system. | 09-13-2012 |
20120233134 | Openstack file deletion - Several different embodiments of a massively scalable object storage system are described. The object storage system is particularly useful for storage in a cloud computing installation whereby shared servers provide resources, software, and data to computers and other devices on demand. In several embodiments, the object storage system includes a ring implementation used to associate object storage commands with particular physical servers such that certain guarantees of consistency, availability, and performance can be met. In other embodiments, the object storage system includes a synchronization protocol used to order operations across a distributed system. In a third set of embodiments, the object storage system includes a metadata management system. In a fourth set of embodiments, the object storage system uses a structured information synchronization system. Features from each set of embodiments can be used to improve the performance and scalability of a cloud computing object storage system. | 09-13-2012 |
20120233228 | APPENDING TO FILES VIA SERVER-SIDE CHUNKING AND MANIFEST MANIPULATION - Several different embodiments of a segmented object storage system are described. The object storage system divides files into a number of object segments, each segment corresponding to a portion of the object, and stores each segment individually in the cloud storage system. The system also generates and stores a manifest file describing the relationship of the various segments to the original data file. Requests to retrieve the segmented file are fulfilled by consulting the manifest file and using the information from the manifest to reconstitute the original data file from the constituent segments. Modifying, appending to, or truncating the object is accomplished by manipulating individual segments and the manifest file. In further embodiments, manipulation of the individual object segments and/or the manifest is used to implement copy-on-write, snapshotting, software transactional memory, and peer-to-peer transmission of the large file. | 09-13-2012 |
20120233251 | Virtual Multi-Cluster Clouds - An improved scalable object storage system allows multiple clusters to work together. Users working with a first cluster, or with a multi-cluster gateway, can ask for services and have the request or data transparently proxied to a second cluster. This gives transparent cross-cluster replication, as well as multi-cluster compute or storage farms based upon spot availability or various provisioning policies. Vendors providing a cloud storage “frontend” can provide multiple backends simultaneously. In one embodiment, a multi-cluster gateway can have a two, three, or higher-level ring that transparently matches an incoming request with the correct cluster. In the ring, a request is first mapped to an abstract “partition” based on a consistent hash function, and then one or more constrained mappings map the partition number to an actual resource. In another embodiment, the multi-cluster gateway is a dumb gateway, and the rings are located only at the cluster level. | 09-13-2012 |
20120233418 | MASSIVELY SCALABLE OBJECT STORAGE - Several different embodiments of a massively scalable object storage system are described. The object storage system is particularly useful for storage in a cloud computing installation whereby shared servers provide resources, software, and data to computers and other devices on demand. In several embodiments, the object storage system includes a ring implementation used to associate object storage commands with particular physical servers such that certain guarantees of consistency, availability, and performance can be met. In other embodiments, the object storage system includes a synchronization protocol used to order operations across a distributed system. In a third set of embodiments, the object storage system includes a metadata management system. In a fourth set of embodiments, the object storage system uses a structured information synchronization system. Features from each set of embodiments can be used to improve the performance and scalability of a cloud computing object storage system. | 09-13-2012 |
20120233463 | Cluster Federation and Trust - An improved scalable object storage system allows multiple clusters to work together. In one embodiment, a trust and federation relationship is established between a first cluster and a second cluster. This is done by designating a first cluster as a trust root. The trust root receives contact from another cluster, and the two clusters exchange cryptographic credentials. The two clusters mutually authenticate each other based upon the credentials, and optionally relative to a third information service, and establish a service connection. Services from the remote cluster are registered as being available to the cluster designated as the trust root. Multi-cluster gateways can also be designated as the trust root, and joined clusters can be mutually untrusting. Two one-way trust and federation relationships can be set up to form a trusted bidirectional channel. | 09-13-2012 |
20120233490 | Openstack synchronization and ordering - Several different embodiments of a massively scalable object storage system are described. The object storage system is particularly useful for storage in a cloud computing installation whereby shared servers provide resources, software, and data to computers and other devices on demand. In several embodiments, the object storage system includes a ring implementation used to associate object storage commands with particular physical servers such that certain guarantees of consistency, availability, and performance can be met. In other embodiments, the object storage system includes a synchronization protocol used to order operations across a distributed system. In a third set of embodiments, the object storage system includes a metadata management system. In a fourth set of embodiments, the object storage system uses a structured information synchronization system. Features from each set of embodiments can be used to improve the performance and scalability of a cloud computing object storage system. | 09-13-2012 |
20120233522 | METHOD FOR HANDLING LARGE OBJECT FILES IN AN OBJECT STORAGE SYSTEM - Several different embodiments of a segmented object storage system are described. The object storage system divides files into a number of object segments, each segment corresponding to a portion of the object, and stores each segment individually in the cloud storage system. The system also generates and stores a manifest file describing the relationship of the various segments to the original data file. Requests to retrieve the segmented file are fulfilled by consulting the manifest file and using the information from the manifest to reconstitute the original data file from the constituent segments. Modifying, appending to, or truncating the object is accomplished by manipulating individual segments and the manifest file. In further embodiments, manipulation of the individual object segments and/or the manifest is used to implement copy-on-write, snapshotting, software transactional memory, and peer-to-peer transmission of the large file. | 09-13-2012 |
20130061084 | Massively Scalable Object Storage - Several different embodiments of a massively scalable object storage system are described. The object storage system is particularly useful for storage in a cloud computing installation whereby shared servers provide resources, software, and data to computers and other devices on demand. In several embodiments, the object storage system includes a ring implementation used to associate object storage commands with particular physical servers such that certain guarantees of consistency, availability, and performance can be met. In other embodiments, the object storage system includes a synchronization protocol used to order operations across a distributed system. In a third set of embodiments, the object storage system includes a metadata management system. In a fourth set of embodiments, the object storage system uses a structured information synchronization system. Features from each set of embodiments can be used to improve the performance and scalability of a cloud computing object storage system. | 03-07-2013 |
20130091241 | Distributed Rate Limiting Of Handling Requests - In one embodiment, the present invention includes a method for performing rate limiting in a horizontally distributed and scalable manner. The method includes receiving a request in a rate limiter. In turn, a sleep time can be obtained for the request based at least in part on a time value and an allotted time per request, and the request can be delayed according to the sleep time. The time value can be obtained from a distributed key value store using a key generated from the request. After this sleep time, the request can be forwarded from the rate limiter to the handling server. | 04-11-2013 |
20140040505 | MASSIVELY SCALABLE OBJECT STORAGE SYSTEM - Several different embodiments of a massively scalable object storage system are described. The object storage system is particularly useful for storage in a cloud computing installation whereby shared servers provide resources, software, and data to computers and other devices on demand. In several embodiments, the object storage system includes a ring implementation used to associate object storage commands with particular physical servers such that certain guarantees of consistency, availability, and performance can be met. In other embodiments, the object storage system includes a synchronization protocol used to order operations across a distributed system. In a third set of embodiments, the object storage system includes a metadata management system. In a fourth set of embodiments, the object storage system uses a structured information synchronization system. Features from each set of embodiments can be used to improve the performance and scalability of a cloud computing object storage system. | 02-06-2014 |
20140082301 | MASSIVELY SCALABLE OBJECT STORAGE FOR STORING OBJECT REPLICAS - An example method for storing data includes providing a plurality of physical storage pools, each storage pool including a plurality of storage nodes coupled to a network. The method also includes mapping a partition of a plurality of partitions to a set of physical storage pools, where each physical storage pool of the set of physical storage pools is located in a different availability zone, and the storage nodes within an availability zone are subject to a correlated loss of access to stored data. The method further includes receiving a data management request over the network, the data management request being associated with a data object. The method also includes identifying a first partition of the plurality of partitions corresponding to the received data management request and manipulating the data object in the physical storage pools mapped to the first partition in accordance with the data management request. | 03-20-2014 |
20140222949 | MASSIVELY SCALABLE OBJECT STORAGE SYSTEM - Several different embodiments of a massively scalable object storage system are described. The object storage system is particularly useful for storage in a cloud computing installation whereby shared servers provide resources, software, and data to computers and other devices on demand. In several embodiments, the object storage system includes a ring implementation used to associate object storage commands with particular physical servers such that certain guarantees of consistency, availability, and performance can be met. In other embodiments, the object storage system includes a synchronization protocol used to order operations across a distributed system. In a third set of embodiments, the object storage system includes a metadata management system. In a fourth set of embodiments, the object storage system uses a structured information synchronization system. Features from each set of embodiments can be used to improve the performance and scalability of a cloud computing object storage system. | 08-07-2014 |
20140304334 | VIRTUAL MULTI-CLUSTER CLOUDS - An improved scalable object storage system includes methods and systems allowing multiple clusters to work together. Users working with a first cluster, or with a multi-cluster gateway, can ask for services and have the request or data transparently proxied to a second cluster. This gives transparent cross-cluster replication, as well as multi-cluster compute or storage farms based upon spot availability or various provisioning policies. Vendors providing a cloud storage “frontend” can provide multiple backends simultaneously. In one embodiment, a multi-cluster gateway can have a two, three, or higher-level ring that transparently matches an incoming request with the correct cluster. In the ring, a request is first mapped to an abstract “partition” based on a consistent hash function, and then one or more constrained mappings map the partition number to an actual resource. In another embodiment, the multi-cluster gateway is a dumb gateway, and the rings are located only at the cluster level. | 10-09-2014 |