Patent application number | Description | Published |
20090138618 | RATE-CONTROLLABLE PEER-TO-PEER DATA STREAM ROUTING - Difficulties associated with choosing advantageous network routes between server and clients are mitigated by a routing system that is devised to use many routing path sets, where respective sets comprise a number of routing paths covering all of the clients, including through other clients. A server may then apportion a data stream among all of the routing path sets. The server may also detect the performance of the computer network while sending the data stream between clients, and may adjust the apportionment of the routing path sets including the route. The clients may also be configured to operate as servers of other data streams, such as in a videoconferencing session, for example, and may be configured to send detected route performance information along with the portions of the various data streams. | 05-28-2009 |
20090198829 | Multi-Rate Peer-Assisted Data Streaming - The described implementations relate to distributed network management and more particularly to enhancing distributed network utility. One technique selects multiple trees to distribute content to multiple receivers in a session where individual receivers can receive the distributed content at one of a plurality of rates. The technique further adjustably allocates content distribution across the multiple trees to increase a sum of utilities of the multiple receivers. | 08-06-2009 |
20090204636 | MULTIMODAL OBJECT DE-DUPLICATION - Various object de-duplication techniques may be applied to object systems (such as to files in a file store) to identify similar or identical objects or portions thereof, so that duplicate objects or object portions may be associated with one copy, and the duplicate copies may be removed. However, an object de-duplication technique that is suitable for de-duplicating one type of object may be inefficient for de-duplicating another type of object; e.g., a de-duplication method that significantly condenses sets of small objects may achieve very little condensation among sets of large objects, and vice versa. A multimodal approach to object de-duplication may be devised that analyzes an object to be stored and chooses a de-duplication technique that is likely to be effective for storing the object. The object index may be configured to support several de-duplication schemes for indexing and storing many types of objects in a space-economizing manner. | 08-13-2009 |
20090303880 | DATA CENTER INTERCONNECT AND TRAFFIC ENGINEERING - A system for commoditizing data center networking is disclosed. The system includes an interconnection topology for a data center having a plurality of servers and a plurality of nodes of a network in the data center through which data packets may be routed. The system uses a routing scheme where the routing is oblivious to the traffic pattern between nodes in the network, and wherein the interconnection topology contains a plurality of paths between one or more servers. The multipath routing may be Valiant load balancing. It disaggregates the function of load balancing into a group of regular servers, with the result that load balancing server hardware can be distributed amongst racks in the data center leading to greater agility and less fragmentation. The architecture creates a huge, flexible switching domain, supporting any server/any service, full mesh agility, and unregimented server capacity at low cost. | 12-10-2009 |
20090307334 | DATA CENTER WITHOUT STRUCTURAL BOTTLENECKS - A method of networking a plurality of servers together within a data center is disclosed. The method includes the step of addressing a data packet for delivery to a destination server by providing the destination server address as a flat address. The method further includes the steps of obtaining routing information required to route the packet to the destination server. This routing information may be obtained from a directory service servicing the plurality of servers. Once the routing information is obtained, the data packet may be routed to the destination server according to the flat address of the destination server and routing information obtained from the directory service. | 12-10-2009 |
20100036903 | DISTRIBUTED LOAD BALANCER - Systems and methods that distribute load balancing functionalities in a data center. A network of demultiplexers and load balancer servers enable a calculated scaling and growth operation, wherein capacity of load balancing operation can be adjusted by changing the number of load balancer servers. Accordingly, load balancing functionality/design can be disaggregated to increase resilience and flexibility for both the load balancing and switching mechanisms of the data center. | 02-11-2010 |
20100080144 | RESILIENT 1:N FIRST-HOP GATEWAY SELECTION MECHANISM - Exemplary methods, systems, and computer program products describe selecting a gateway based on health and performance information of a plurality of gateways. The techniques describe gateways advertising health and performance information, computing devices creating a table of this health and performance information, and selecting a gateway using the table. In response to changes in the health and performance information, the computing device may select a different gateway. The process allows network traffic load to be distributed across a plurality of gateways. This process further provides resilience by allowing a plurality of active gateways to substitute for a non-functioning gateway. | 04-01-2010 |
20100082838 | ISP-FRIENDLY RATE ALLOCATION FOR P2P APPLICATIONS - An ISP-friendly rate allocation system and method that reduces network traffic across ISP boundaries in a peer-to-peer (P2P) network, Embodiments of the system and method continuously solve a global optimization problem and dictate accordingly how much bandwidth is allocated on each connection. Embodiments of the system and method minimize load on a server in communication with the P2P network, minimize ISP-unfriendly traffic while keeping the minimum server load unaffected, and maximize peer prefetching. Two different techniques are used to compute rate allocation, including a utility function optimization technique and a minimum cost flow formulation technique. The utility function optimization technique constructs a utility function and optimizes that utility function. The minimum cost flow formulation technique generates a minimum cost flow formulation using a bipartite graph have a vertices set and an edges set. A distributed minimum cost flow formulation is solved using Lagrangian multipliers. | 04-01-2010 |
20100085979 | MODELS FOR ROUTING TREE SELECTION IN PEER-TO-PEER COMMUNICATIONS - Peer-to-peer communications sessions involve the transmission of one or more data streams from a source to a set of receivers that may redistribute portions of the data stream via a set of routing trees. Achieving a comparatively high, sustainable data rate throughput of the data stream(s) may be difficult due to the large number of available routing trees, as well as pertinent variations in the nature of the communications session (e.g., upload communications caps, network link caps, the presence or absence of helpers, and the full or partial interconnectedness of the network.) The selection of routing trees may be facilitated through the representation of the node set according to a linear programming model, such as a primal model or a linear programming dual model, and iterative processes for applying such models and identifying low-cost routing trees during an iteration. | 04-08-2010 |
20100146108 | RATE-CONTROLLABLE PEER-TO-PEER DATA STREAM ROUTING - Difficulties associated with choosing advantageous network routes between server and clients are mitigated by a routing system that is devised to use many routing path sets, where respective sets comprise a number of routing paths covering all of the clients, including through other clients. A server may then apportion a data stream among all of the routing path sets. The server may also detect the performance of the computer network while sending the data stream between clients, and may adjust the apportionment of the routing path sets including the route. The clients may also be configured to operate as servers of other data streams, such as in a videoconferencing session, for example, and may be configured to send detected route performance information along with the portions of the various data streams. | 06-10-2010 |
20100195488 | OPTIMIZED TRANSPORT PROTOCOL FOR DELAY-SENSITIVE DATA - Transmission delays are minimized when packets are transmitted from a source computer over a network to a destination computer. The source computer measures the network's available bandwidth, forms a sequence of output packets from a sequence of data packets, and transmits the output packets over the network to the destination computer, where the transmission rate is ramped up to the measured bandwidth. In conjunction with the transmission, the source computer monitors a transmission delay indicator which it computes using acknowledgement packets it receives from the destination computer. Whenever the indicator specifies that the transmission delay is increasing, the source computer reduces the transmission rate until the indicator specifies that the delay is unchanged. The source computer dynamically decides whether each output packet will be a forward error correction packet or a single data packet, where the decision is based on minimizing the expected transmission delays. | 08-05-2010 |
20100306408 | AGILE DATA CENTER NETWORK ARCHITECTURE - This patent application relates to an agile network architecture that can be employed in data centers, among others. One implementation provides a virtual layer-2 network connecting machines of a layer-3 infrastructure. | 12-02-2010 |
20100332579 | PROXIMITY GUIDED DATA DISCOVERY - Techniques are described for sharing content among peers. Locality domains are treated as first order network units. Content is located at the level of a locality domain using a hierarchical DHT in which nodes correspond to locality domains. A peer searches for a given piece of content in a proximity guided manner and terminates at the earliest locality domain (in the hierarchy) which has the content. Locality domains are organized into hierarchical clusters based on their proximity. | 12-30-2010 |
20110216648 | CONGESTION CONTROL FOR DELAY SENSITIVE APPLICATIONS - In various embodiments, methods and systems are disclosed for a hybrid rate plus window based congestion protocol that controls the rate of packet transmission into the network and provides low queuing delay, practically zero packet loss, fair allocation of network resources amongst multiple flows, and full link utilization. In one embodiment, a congestion window may be used to control the maximum number of outstanding bits, a transmission rate may be used to control the rate of packets entering the network (packet pacing), a queuing delay based rate update may be used to control queuing delay within tolerated bounds and minimize packet loss, and aggressive ramp-up/graceful back-off may be used to fully utilize the link capacity and additive-increase, multiplicative-decrease (AIMD) rate control may be used to provide fairness amongst multiple flows. | 09-08-2011 |
20110276744 | FLASH MEMORY CACHE INCLUDING FOR USE WITH PERSISTENT KEY-VALUE STORE - Described is using flash memory, RAM-based data structures and mechanisms to provide a flash store for caching data items (e.g., key-value pairs) in flash pages. A RAM-based index maps data items to flash pages, and a RAM-based write buffer maintains data items to be written to the flash store, e.g., when a full page can be written. A recycle mechanism makes used pages in the flash store available by destaging a data item to a hard disk or reinserting it into the write buffer, based on its access pattern. The flash store may be used in a data deduplication system, in which the data items comprise chunk-identifier, metadata pairs, in which each chunk-identifier corresponds to a hash of a chunk of data that indicates. The RAM and flash are accessed with the chunk-identifier (e.g., as a key) to determine whether a chunk is a new chunk or a duplicate. | 11-10-2011 |
20110276780 | Fast and Low-RAM-Footprint Indexing for Data Deduplication - The subject disclosure is directed towards a data deduplication technology in which a hash index service's index maintains a hash index in a secondary storage device such as a hard drive, along with a compact index table and look-ahead cache in RAM that operate to reduce the I/O to access the secondary storage device during deduplication operations. Also described is a session cache for maintaining data during a deduplication session, and encoding of a read-only compact index table for efficiency. | 11-10-2011 |
20110276781 | Fast and Low-RAM-Footprint Indexing for Data Deduplication - The subject disclosure is directed towards a data deduplication technology in which a hash index service's index maintains a hash index in a secondary storage device such as a hard drive, along with a compact index table and look-ahead cache in RAM that operate to reduce the I/O to access the secondary storage device during deduplication operations. Also described is a session cache for maintaining data during a deduplication session, and encoding of a read-only compact index table for efficiency. | 11-10-2011 |
20120102298 | Low RAM Space, High-Throughput Persistent Key-Value Store using Secondary Memory - Described is using flash memory (or other secondary storage), RAM-based data structures and mechanisms to access key-value pairs stored in the flash memory using only a low RAM space footprint. A mapping (e.g. hash) function maps key-value pairs to a slot in a RAM-based index. The slot includes a pointer that points to a bucket of records on flash memory that each had keys that mapped to the slot. The bucket of records is arranged as a linear-chained linked list, e.g., with pointers from the most-recently written record to the earliest written record. Also described are compacting non-contiguous records of a bucket onto a single flash page, and garbage collection. Still further described is load balancing to reduce variation in bucket sizes, using a bloom filter per slot to avoid unnecessary searching, and splitting a slot into sub-slots. | 04-26-2012 |
20120166401 | Using Index Partitioning and Reconciliation for Data Deduplication - The subject disclosure is directed towards a data deduplication technology in which a hash index service's index is partitioned into subspace indexes, with less than the entire hash index service's index cached to save memory. The subspace index is accessed to determine whether a data chunk already exists or needs to be indexed and stored. The index may be divided into subspaces based on criteria associated with the data to index, such as file type, data type, time of last usage, and so on. Also described is subspace reconciliation, in which duplicate entries in subspaces are detected so as to remove entries and chunks from the deduplication system. Subspace reconciliation may be performed at off-peak time, when more system resources are available, and may be interrupted if resources are needed. Subspaces to reconcile may be based on similarity, including via similarity of signatures that each compactly represents the subspace's hashes. | 06-28-2012 |
20120166448 | Adaptive Index for Data Deduplication - The subject disclosure is directed towards a data deduplication technology in which a hash index service's index and/or indexing operations are adaptable to balance deduplication performance savings, throughput and resource consumption. The indexing service may employ hierarchical chunking using different levels of granularity corresponding to chunk size, a sampled compact index table that contains compact signatures for less than all of the hash index's (or subspace's) hash values, and/or selective subspace indexing based on similarity of a subspace's data to another subspace's data and/or to incoming data chunks. | 06-28-2012 |
20130054544 | Content Aware Chunking for Achieving an Improved Chunk Size Distribution - The subject disclosure is directed towards partitioning a file into chunks that satisfy a chunk size restriction, such as maximum and minimum chunk sizes, using a sliding window. For file positions within the chunk size restriction, a signature representative of a window fingerprint is compared with a target pattern, with a chunk boundary candidate identified if matched. Other signatures and patterns are then checked to determine a highest ranking signature (corresponding to a lowest numbered Rule) to associate with that chunk boundary candidate, or set an actual boundary if the highest ranked signature is matched. If the maximum chunk size is reached without matching the highest ranked signature, the chunking mechanism regresses to set the boundary based on the candidate with the next highest ranked signature (if no candidates, the boundary is set at the maximum). Also described is setting chunk boundaries based upon pattern detection (e.g., runs of zeros). | 02-28-2013 |
20130263151 | Consistent Hashing Table for Workload Distribution - Described is a technology by which a consistent hashing table of bins maintains values representing nodes of a distributed system. An assignment stage uses a consistent hashing function and a selection algorithm to assign values that represent the nodes to the bins. In an independent mapping stage, a mapping mechanism deterministically maps an object identifier/key to one of the bins as a mapped-to bin. | 10-03-2013 |
20130279338 | CONGESTION CONTROL FOR DELAY SENSITIVE APPLICATIONS - In various embodiments, methods and systems are disclosed for a hybrid rate plus window based congestion protocol that controls the rate of packet transmission into the network and provides low queuing delay, practically zero packet loss, fair allocation of network resources amongst multiple flows, and full link utilization. In one embodiment, a congestion window may be used to control the maximum number of outstanding bits, a transmission rate may be used to control the rate of packets entering the network (packet pacing), a queuing delay based rate update may be used to control queuing delay within tolerated bounds and minimize packet loss, and aggressive ramp-up/graceful back-off may be used to fully utilize the link capacity and additive-increase, multiplicative-decrease (AIMD) rate control may be used to provide fairness amongst multiple flows. | 10-24-2013 |
20130282964 | FLASH MEMORY CACHE INCLUDING FOR USE WITH PERSISTENT KEY-VALUE STORE - Described is using flash memory, RAM-based data structures and mechanisms to provide a flash store for caching data items (e.g., key-value pairs) in flash pages. A RAM-based index maps data items to flash pages, and a RAM-based write buffer maintains data items to be written to the flash store, e.g., when a full page can be written. A recycle mechanism makes used pages in the flash store available by destaging a data item to a hard disk or reinserting it into the write buffer, based on its access pattern. The flash store may be used in a data deduplication system, in which the data items comprise chunk-identifier, metadata pairs, in which each chunk-identifier corresponds to a hash of a chunk of data that indicates. The RAM and flash are accessed with the chunk-identifier (e.g., as a key) to determine whether a chunk is a new chunk or a duplicate. | 10-24-2013 |
20130282965 | FLASH MEMORY CACHE INCLUDING FOR USE WITH PERSISTENT KEY-VALUE STORE - Described is using flash memory, RAM-based data structures and mechanisms to provide a flash store for caching data items (e.g., key-value pairs) in flash pages. A RAM-based index maps data items to flash pages, and a RAM-based write buffer maintains data items to be written to the flash store, e.g., when a full page can be written. A recycle mechanism makes used pages in the flash store available by destaging a data item to a hard disk or reinserting it into the write buffer, based on its access pattern. The flash store may be used in a data deduplication system, in which the data items comprise chunk-identifier, metadata pairs, in which each chunk-identifier corresponds to a hash of a chunk of data that indicates. The RAM and flash are accessed with the chunk-identifier (e.g., as a key) to determine whether a chunk is a new chunk or a duplicate. | 10-24-2013 |
20130301606 | LOAD AWARE RESOURCE ALLOCATION IN WIRELESS NETWORKS - A technique for resource allocation in a wireless network (for example, an access point type wireless network), which supports concurrent communication on a band of channels, is provided. The technique includes accepting connectivity information for the network that supports concurrent communication on the band of channels. A conflict graph is generated from the connectivity information. The generated conflict graph models concurrent communication on the band of channels. A linear programming approach, which incorporates information form the conflict graph and rate requirements for nodes of the network, can be utilized to maximize throughput of the network. | 11-14-2013 |
20130346672 | Multi-Tiered Cache with Storage Medium Awareness - The subject disclosure is directed towards a multi-tiered cache having cache tiers with different access properties. Objects are written to a selected a tier of the cache based upon object-related properties and/or cache-related properties. In one aspect, objects are stored in an active log among a plurality of logs. The active log is sealed upon reaching a target size, with a new active log opened. Garbage collecting is performed on a sealed log, such as the sealed log with the most garbage therein. | 12-26-2013 |
20130346725 | STRUCTURING STORAGE BASED ON LATCH-FREE B-TREES - A request to modify an object in storage that is associated with one or more computing devices may be obtained, the storage organized based on a latch-free B-tree structure. A storage address of the object may be determined, based on accessing a mapping table that includes map indicators mapping logical object identifiers to physical storage addresses. A prepending of a first delta record to a prior object state of the object may be initiated, the first delta record indicating an object modification associated with the obtained request. Installation of a first state change associated with the object modification may be initiated via a first atomic operation on a mapping table entry that indicates the prior object state of the object. For example, the latch-free B-tree structure may include a B-tree like index structure over records as the objects, and logical page identifiers as the logical object identifiers. | 12-26-2013 |
20140189348 | Integrated Data Deduplication and Encryption - The subject disclosure is directed towards encryption and deduplication integration between computing devices and a network resource. Files are partitioned into data blocks and deduplicated via removal of duplicate data blocks. Using multiple cryptographic keys, each data block is encrypted and stored at the network resource but can only be decrypted by an authorized user, such as domain entity having an appropriate deduplication domain-based cryptographic key. Another cryptographic key referred to as a content-derived cryptographic key ensures that duplicate data blocks encrypt to substantially equivalent encrypted data. | 07-03-2014 |
20140244604 | PREDICTING DATA COMPRESSIBILITY USING DATA ENTROPY ESTIMATION - The subject disclosure is directed towards predicting compressibility of a data block, and using the predicted compressibility in determining whether a data block if compressed will be sufficiently compressible to justify compression. In one aspect, data of the data block is processed to obtain an entropy estimate of the data block, e.g., based upon distinct value estimation. The compressibility prediction may be used in conjunction with a chunking mechanism of a data deduplication system. | 08-28-2014 |
20140280664 | CACHING CONTENT ADDRESSABLE DATA CHUNKS FOR STORAGE VIRTUALIZATION - The subject disclosure is directed towards using primary data deduplication concepts for more efficient access of data via content addressable caches. Chunks of data, such as deduplicated data chunks, are maintained in a fast access client-side cache, such as containing chunks based upon access patterns. The chunked content is content addressable via a hash or other unique identifier of that content in the system. When a chunk is needed, the client-side cache (or caches) is checked for the chunk before going to a file server for the chunk. The file server may likewise maintain content addressable (chunk) caches. Also described are cache maintenance, management and organization, including pre-populating caches with chunks, as well as using RAM and/or solid-state storage device caches. | 09-18-2014 |
20140280669 | Memory Sharing Over A Network - Memory is shared among physically distinct, networked computing devices. Each computing device comprises a Remote Memory Interface (RMI) accepting commands from locally executing processes and translating such commands into forms transmittable to a remote computing device. The RMI also accepts remote communications directed to it and translates those into commands directed to local memory. The amount of storage capacity shared is informed by a centralized controller, either a single controller, a hierarchical collection of controllers, or a peer-to-peer negotiation. Requests that are directed to remote high-speed non-volatile storage media are detected or flagged and the process generating the request is suspended such that it can be efficiently revived. The storage capacity provided by remote memory is mapped into the process space of processes executing locally. | 09-18-2014 |
20140379991 | LATCH-FREE, LOG-STRUCTURED STORAGE FOR MULTIPLE ACCESS METHODS - A data manager may include a data opaque interface configured to provide, to an arbitrarily selected page-oriented access method, interface access to page data storage that includes latch-free access to the page data storage. In another aspect, a swap operation may be initiated, of a portion of a first page in cache layer storage to a location in secondary storage, based on initiating a prepending of a partial swap delta record to a page state associated with the first page, the partial swap delta record including a main memory address indicating a storage location of a flush delta record that indicates a location in secondary storage of a missing part of the first page. In another aspect, a page manager may initiate a flush operation of a first page in cache layer storage to a location in secondary storage, based on atomic operations with flush delta records. | 12-25-2014 |