Patent application number | Description | Published |
20090319566 | Data Quality Tracking - In general, a method includes determining metric values associated with data quality for one or more child nodes. Metric values are determined for a parent node based on the metric values of at least some of the child nodes, and relationships between one or more parent nodes and one or more child nodes define a hierarchy. The determination of the metric value for the parent node is repeated for multiple instances. | 12-24-2009 |
20100138388 | MAPPING INSTANCES OF A DATASET WITHIN A DATA MANAGEMENT SYSTEM - Mapping data stored in a data storage system for use by a computer system includes processing specifications of dataflow graphs that include nodes representing computations interconnected by links representing flows of data. At least one of the dataflow graphs receives a flow of data from at least one input dataset and at least one of the dataflow graphs provides a flow of data to at least one output dataset. A mapper identifies one or more sets of datasets. Each dataset in a given set matches one or more criteria for identifying different versions of a single dataset. A user interface is provided to receive a mapping between at least two datasets in a given set. The mapping received over the user interface is stored in association with a dataflow graph that provides data to or receives data from the datasets of the mapping. | 06-03-2010 |
20100138420 | VISUALIZING RELATIONSHIPS BETWEEN DATA ELEMENTS - In general, a specification of multiple contexts that are related according to a hierarchy is received. Relationships are determined among three or more metadata objects, and at least some of the metadata objects are grouped into one or more respective groups. Each of at least some of the groups is based on a selected one of the contexts and is represented by a node in a diagram. Relationships among the nodes are determined based on the relationships among the metadata objects in the groups represented by the nodes, and a visual representation is generated of the diagram including the nodes and the relationships among the nodes. | 06-03-2010 |
20100211953 | MANAGING TASK EXECUTION - Managing task execution includes: receiving a specification of a plurality of tasks to be performed by respective functional modules; processing a flow of input data using a dataflow graph that includes nodes representing data processing components connected by links representing flows of data between data processing components; in response to at least one flow of data provided by at least one data processing component, generating a flow of messages; and in response to each of the messages in the flow of messages, performing an iteration of a set of one or more tasks using one or more corresponding functional modules. | 08-19-2010 |
20110016354 | FAULT TOLERANT BATCH PROCESSING - Among other aspects disclosed are a method and system for processing a batch of input data in a fault tolerant manner. The method includes reading a batch of input data including a plurality of records from one or more data sources and passing the batch through a dataflow graph. The dataflow graph includes two or more nodes representing components connected by links representing flows of data between the components. At least one but fewer than all of the components includes a checkpoint process for an action performed for each of multiple units of work associated with one or more of the records. The checkpoint process includes opening a checkpoint buffer stored in non-volatile memory at the start of processing for the batch. For each unit of work from the batch, if a result from performing the action for the unit of work was previously saved in the checkpoint buffer, the saved result is used to complete processing of the unit of work without performing the action again. If a result from performing the action for the unit of work is not saved in the checkpoint buffer. The action is performed to complete processing of the unit of work and the result from performing the action is saved in the checkpoint buffer. | 01-20-2011 |
20110145748 | SPECIFYING USER INTERFACE ELEMENTS - Providing a user interface for configuring a computer-executable application includes receiving a specification defining: relationships among user interface elements, the relationships based on dependencies between components of a dataflow graph that includes multiple nodes representing components of the dataflow graph and links between the nodes representing flows of data between the components, parameters defining respective characteristics of the components of the dataflow graph, and variables defining respective characteristics of the user interface elements. During operation of a user interface, user interface elements are displayed based on the relationships defined in the specification. | 06-16-2011 |
20110153711 | MANAGING OBJECTS USING A CLIENT-SERVER BRIDGE - A method for supporting communication between a client and a server includes receiving a first message from a client. The method also includes creating an object in response to the first message. The method also includes sending a response to the first message to the client. The method also includes receiving changes to the object from a server. The method also includes storing the changes to the object. The method also includes receiving a second message from the client. The method also includes sending the stored changes to the client with a response to the second message. | 06-23-2011 |
20120311588 | FAULT TOLERANT BATCH PROCESSING - Among other aspects disclosed are a method and system for processing a batch of input data in a fault tolerant manner. The method includes reading a batch of input data including a plurality of records from one or more data sources and passing the batch through a dataflow graph. The dataflow graph includes two or more nodes representing components connected by links representing flows of data between the components. At least one but fewer than all of the components includes a checkpoint process for an action performed for each of multiple units of work associated with one or more of the records. The checkpoint process includes opening a checkpoint buffer stored in non-volatile memory at the start of processing for the batch. | 12-06-2012 |
20140053159 | FAULT TOLERANT BATCH PROCESSING - Among other aspects disclosed are a method and system for processing a batch of input data in a fault tolerant manner. The method includes reading a batch of input data including a plurality of records from one or more data sources and passing the batch through a dataflow graph. The dataflow graph includes two or more nodes representing components connected by links representing flows of data between the components. At least one but fewer than all of the components includes a checkpoint process for an action performed for each of multiple units of work associated with one or more of the records. The checkpoint process includes opening a checkpoint buffer stored in non-volatile memory at the start of processing for the batch. | 02-20-2014 |
20140143757 | DYNAMIC COMPONENT PERFORMANCE MONITORING - Methods, systems, and apparatus, including computer programs encoded on computer storage media, for dynamic graph performance monitoring. One of the methods includes receiving input data by the data processing system, the input data provided by an application executing on the data processing system. The method includes determining a characteristic of the input data. The method includes identifying, by the application, a dynamic component from multiple available dynamic components based on the determined characteristic, the multiple available dynamic components being stored in a data storage system. The method includes processing the input data using the identified dynamic component. The method also includes determining one or more performance metrics associated with the processing. | 05-22-2014 |
20140143760 | DYNAMIC GRAPH PERFORMANCE MONITORING - Methods, systems, and apparatus, including computer programs encoded on computer storage media, for dynamic graph performance monitoring. One of the methods includes receiving multiple units of work that each include one or more work elements. The method includes determining a characteristic of the first unit of work. The method includes identifying, by a component of the first dataflow graph, a second dataflow graph from multiple available dataflow graphs based on the determined characteristic, the multiple available dataflow graphs being stored in a data storage system. The method includes processing the first unit of work using the second dataflow graph. The method includes determining one or more performance metrics associated with the processing. | 05-22-2014 |
20140164495 | MANAGING OBJECTS USING A CLIENT-SERVER BRIDGE - A method for supporting communication between a client and a server includes receiving a first message from a client. The method also includes creating an object in response to the first message. The method also includes sending a response to the first message to the client. The method also includes receiving changes to the object from a server. The method also includes storing the changes to the object. The method also includes receiving a second message from the client. The method also includes sending the stored changes to the client with a response to the second message. | 06-12-2014 |