Patent application number | Description | Published |
20090030863 | TRANSACTIONAL GRAPH-BASED COMPUTATION WITH ERROR HANDLING - Processing transactions using graph-based computations includes determining that at least one of a plurality of graph elements of a computation graph of a set of one or more computation graphs includes a computation to be performed for a given transaction, associating the given transaction with an instance of the computation graph that includes reusable computation elements associated with respective graph elements, and executing the graph to perform the computation. | 01-29-2009 |
20090083313 | Managing Data Flows in Graph-Based Computations - Processing data elements associated with a hierarchy that includes multiple levels includes: forming multiple flows of data elements, with each flow being associated with a level of the hierarchy; and including within at least some of the flows hierarchical structure information associated with a level of the hierarchy. For at least one of the levels, two or more and fewer than all of the flows include hierarchical structure information associated with that level. | 03-26-2009 |
20090089552 | Dependency Graph Parameter Scoping - A number of tasks are defined according to a dependency graph. Multiple parameter contexts are maintained, each associated with a different scope of the tasks. A parameter used in a first of the tasks is bound to a value. This binding includes identifying a first of the contexts according to the dependency graph and retrieving the value for the parameter from the identified context. | 04-02-2009 |
20090287986 | MANAGING STORAGE OF INDIVIDUALLY ACCESSIBLE DATA UNITS - A method includes determining a length of a file and storing the length of the file in a first memory location. An endpoint of a last complete record within the file is determined and the endpoint is stored in a second memory location. The length of the file stored in the first memory location is compared to a current length of the file, and a data structure associated with the file is updated beginning at the endpoint if the current length of the file exceeds the length of the file stored in the first memory location. | 11-19-2009 |
20100122048 | MANAGING MEMORY PAGES - A method, and corresponding software and system, is described for paging memory used for one or more sequentially-accessed data structure. The method includes providing a data structure representing an order in which memory pages are to be reused; and maintaining the data structure according to a history of access to a memory page associated with one of the sequentially-accessed data structures. A position of the memory page in the order depends on a transition of sequential access off of the memory page. | 05-13-2010 |
20100169274 | MANAGING STORAGE OF INDIVIDUALLY ACCESSIBLE DATA UNITS - Managing data includes: receiving at least one group of individually accessible data units over an input device or port, each data unit identified by a key value, with key values of the received data units being sorted such that the key value identifying a given first data unit that is received before a given second data unit occurs earlier in a sort order than the key value identifying the given second data unit; and processing the data units for storage in a data storage system. The processing includes: storing a plurality of blocks of data, each of one or more of the blocks being generated by combining a plurality of the data units; providing an index that includes an entry for each of the blocks, wherein one or more of the entries enable location, based on a provided key value, of a block that includes data units corresponding to a range of key values that includes the provided key value; and generating one or more screening data structures associated with the stored blocks for determining a possibility that a data unit that includes a given key value was included in the group of individually accessible data units. | 07-01-2010 |
20110078214 | MANAGING MESSAGE QUEUES - A method, and corresponding system and software, is described for writing data to a plurality of queues, each portion of the data being written to a corresponding one of the queues. The method includes, without requiring concurrent locking of more than one queue, determining if a space is available in each queue for writing a corresponding portion of the data, and if available, reserving the spaces in the queues. The method includes writing each portion of the data to a corresponding one of the queues. | 03-31-2011 |
20110078500 | PROCESSING TRANSACTIONS IN GRAPH-BASED APPLICATIONS - A graph-based computation has a plurality of nodes representing graph components connected by one or more links representing data flows between the components. Preparing the computation includes: identifying at least a first set of components to process a plurality of transactions in the computation; and associating with a first transaction a first group of one or more data records and data operations corresponding to the data records, and associating with a second transaction a second group of one or more data records and data operations corresponding to the data records. Executing the computation to process at least the first and second transactions includes delaying execution by the first set of components of a data operation corresponding to an initial data record of the second group until after execution by the first set of components of a data operation corresponding to a final data record of the first group. | 03-31-2011 |
20110093433 | MANAGING METADATA FOR GRAPH-BASED COMPUTATIONS - Determining metadata associated with a graph-based computation includes functionally transforming metadata associated with a first portion of a graph to generate transformed metadata associated with a second portion of the graph; determining a third portion of the graph related to the second portion of the graph; and propagating the transformed metadata from the second portion of the graph to the third portion of the graph. | 04-21-2011 |
20110153662 | MANAGING QUERIES - Managing queries performed on one or more data sources includes: storing at least a first query in a storage medium; selecting the first query for processing; instructing a query engine to process the first query on a first portion of data in the one or more data sources for a first query interval; receiving result data from the query engine based on processing the first query on the first portion of data; saving a state of the first query in the storage medium after the first query interval; instructing the query engine to process a second query during a second query interval after the first query interval; and instructing the query engine to process the first query on a second portion of data in the one or more data sources during a third query interval after the second query interval. | 06-23-2011 |
20110196844 | MANAGING STORAGE OF INDIVIDUALLY ACCESSIBLE DATA UNITS - Managing data includes: receiving at least one group of individually accessible data units over an input device or port, each data unit identified by a key value, with key values of the received data units being sorted such that the key value identifying a given first data unit that is received before a given second data unit occurs earlier in a sort order than the key value identifying the given second data unit; and processing the data units for storage in a data storage system. The processing includes: storing a plurality of blocks of data, each of one or more of the blocks being generated by combining a plurality of the data units; providing an index that includes an entry for each of the blocks, wherein one or more of the entries enable location, based on a provided key value, of a block that includes data units corresponding to a range of key values that includes the provided key value; and generating one or more screening data structures associated with the stored blocks for determining a possibility that a data unit that includes a given key value was included in the group of individually accessible data units. | 08-11-2011 |
20110307897 | DYNAMICALLY LOADING GRAPH-BASED COMPUTATIONS - Processing data includes: receiving units of work that each include one or more work elements, and processing a first unit of work using a first compiled dataflow graph ( | 12-15-2011 |
20120102029 | MANAGING DATA SET OBJECTS - Managing data set objects for graph-based data processing includes: storing a group of one or more data set objects in a data storage system, the data set objects each representing a respective data set; and generating an association between at least a first data set object in the group and at least a first node of a dataflow graph for processing data in a data processing system, the first node representing a source or sink of data in a flow of data represented by a link in the dataflow graph, and the first data set object including a plurality of modes in which different transformational logic is applied to data processed by the first node. | 04-26-2012 |
20120215992 | Sorting - Systems and techniques are disclosed that include in one aspect a computer implemented method storing a received stream of data elements in a buffer, applying a boundary condition to the data elements stored in the buffer after receiving each individual data element of the stream of data elements, and producing one or more data elements from the buffer based on the boundary condition as an output stream of data elements sorted according to a predetermined order. | 08-23-2012 |
20120215997 | MANAGING BUFFER CONDITIONS - Systems and techniques include, in some implementations, a computer implemented method storing a portion of data elements present in a first buffer in a second buffer in response to detecting an overflow condition of the first buffer, wherein the data elements in the first buffer are sorted according to a predetermined order, and inserting a proxy data element in the first buffer to represent the portion of data elements stored to the second buffer. | 08-23-2012 |
20120271862 | MANAGING STORAGE OF INDIVIDUALLY ACCESSIBLE DATA UNITS - Managing data by: receiving a group of individually accessible data units, each data unit identified by a key value, with key values determined such that the key value identifying a first data unit received before a second data unit occurs earlier in a sort order than the key value identifying the second data unit; and processing the data units for storage in a data storage system. The processing includes: storing blocks of data, the blocks being generated by combining a plurality of the data units; providing an index with entries that enable location, based on a provided key value, of a block that includes a data unit corresponding to the provided key value; and generating one or more screening data structures associated with the blocks for determining, based on a given key value, whether to search the stored blocks for a data unit corresponding to the given key value. | 10-25-2012 |
20130013605 | Managing Storage of Data for Range-Based Searching - In general, a value of a numerical attribute of a record stored in a data structure is received. A numerical range is generated that includes the value of the numerical attribute. An entry is stored, in an index associated with the data structure, that specifies a location of the record within the data structure and that includes a first index key and a second index key. The first index key corresponds to a value of an attribute of the record different from the numerical attribute, and the second index key corresponds to the generated numerical range. | 01-10-2013 |
20130013606 | Managing Storage of Data for Range-Based Searching - In general, a value of a numerical attribute of a record stored in a data structure is received. A numerical range is generated that includes the value of the numerical attribute. An entry is stored, in an index associated with the data structure, that specifies a location of the record within the data structure and that includes a first index key and a second index key. The first index key corresponds to a value of an attribute of the record different from the numerical attribute, and the second index key corresponds to the generated numerical range. | 01-10-2013 |
20130297562 | MANAGING METADATA FOR GRAPH-BASED COMPUTATIONS - Determining metadata associated with a graph-based computation includes functionally transforming metadata associated with a first portion of a graph to generate transformed metadata associated with a second portion of the graph; determining a third portion of the graph related to the second portion of the graph; and propagating the transformed metadata from the second portion of the graph to the third portion of the graph. | 11-07-2013 |
20130297904 | MANAGING BUFFER CONDITIONS THROUGH SORTING - Systems and techniques include, in some implementations, a computer implemented method storing a portion of data elements present in a first buffer in a second buffer in response to detecting an overflow condition of the first buffer, wherein the data elements in the first buffer are sorted according to a predetermined order, and inserting a proxy data element in the first buffer to represent the portion of data elements stored to the second buffer. | 11-07-2013 |
20140025685 | MANAGING STORAGE OF INDIVIDUALLY ACCESSIBLE DATA UNITS - Managing data by: receiving a group of individually accessible data units, each data unit identified by a key value, with key values determined such that the key value identifying a first data unit received before a second data unit occurs earlier in a sort order than the key value identifying the second data unit; and processing the data units for storage in a data storage system. The processing includes: storing blocks of data, the blocks being generated by combining a plurality of the data units; providing an index with entries that enable location, based on a provided key value, of a block that includes a data unit corresponding to the provided key value; and generating one or more screening data structures associated with the blocks for determining, based on a given key value, whether to search the stored blocks for a data unit corresponding to the given key value. | 01-23-2014 |
20140032617 | MAPPING ENTITIES IN DATA MODELS - Mapping information that specifies attributes of destination entities in terms of attributes of source entities is received. At least some source entities correspond to respective sets of records in one or more data storage systems. The mapping information is processed to generate a procedural specification for computing values corresponding to attributes of destination entities. Collections of nodes are generated, each including a first node representing a first relational expression associated with a specified attribute. At least some collections form a directed acyclic graph that includes links to one or more other nodes representing respective relational expressions associated with at least one attribute of at least one source entity referenced by a relational expression of a node in the graph. At least two of the collections are merged with each other to form a third collection based on comparing relational expressions of nodes being merged. | 01-30-2014 |
20140317629 | CONTROLLING TASKS PERFORMED BY A COMPUTING SYSTEM - Controlling tasks includes: receiving ordering information that specifies at least a partial ordering among a plurality of tasks; and generating instructions for performing at least some of the tasks based at least in part on the ordering information. Instructions are stored for executing a first subroutine corresponding to a first task, including a first control section that controls execution of at least a second subroutine corresponding to a second task, the first control section including a function configured to change state information associated with the second task, and to determine whether or not to initiate execution of the second subroutine based on the changed state information. Instructions are stored for executing the second subroutine, including a task section for performing the second task and a second control section that controls execution of a third subroutine corresponding to a third task. | 10-23-2014 |
20140317632 | CONTROLLING TASKS PERFORMED BY A COMPUTING SYSTEM - A graph-based program specification specifies at least a partial ordering among a plurality of tasks represented by its nodes. Executing a specified program includes: executing a first subroutine corresponding to a first task, including a first task section for performing the first task; storing state information indicating a state of the first task selected from a set of possible states that includes: a pending state in which the first task section is waiting to perform the first task, and a suppressed state in which the first task section has been prevented from performing the first task; and executing a second subroutine corresponding to a second task, including a second task section for performing the second task, and a control section that controls execution of the second task section based at least in part on the state of the first task indicated by the stored state information. | 10-23-2014 |
20150043579 | MANAGING DATA FEEDS - Managing data units broadcast from a data feed, without requiring re-transmission by a source of the data feed, includes: at a first node in a network, receiving at least a portion of a data feed including a plurality of data units; at a second node in the network, receiving at least a portion of the data feed; identifying an interruption in receiving the data feed at the first node; determining an extent of a data lacuna extending between a last data unit received by the first node prior to the interruption and a first data unit received by the first node after the interruption; and sending a request from the first node for results saved by the second node, the results saved by the second node corresponding to the data lacuna. | 02-12-2015 |