Patent application number | Description | Published |
20100030896 | ESTIMATING LATENCIES FOR QUERY OPTIMIZATION IN DISTRIBUTED STREAM PROCESSING - A “Query Optimizer” provides a cost estimation metric referred to as “Maximum Accumulated Overload” (MAO). MAO is approximately equivalent to maximum system latency in a data stream management system (DSMS). Consequently, MAO is directly relevant for use in optimizing latencies in real-time streaming applications running multiple continuous queries (CQs) over high data-rate event sources. In various embodiments, the Query Optimizer computes MAO given knowledge of original operator statistics, including “operator selectivity” and “cycles/event” in combination with an expected event arrival workload. Beyond use in query optimization to minimize worst-case latency, MAO is useful for addressing problems including admission control, system provisioning, user latency reporting, operator placements (in a multi-node environment), etc. In addition, MAO, as a surrogate for worst-case latency, is generally applicable beyond streaming systems, to any queue-based workflow system with control over the scheduling strategy. | 02-04-2010 |
20110282812 | DYNAMIC PATTERN MATCHING OVER ORDERED AND DISORDERED DATA STREAMS - Architecture introduces a new pattern operator referred to as called an augmented transition network (ATN), which is a streaming adaptation of non-reentrant, fixed-state ATNs for dynamic patterns. Additional user-defined information is associated with automaton states and is accessible to transitions during execution. ATNs are created that directly model complex pattern continuous queries with arbitrary cycles in a transition graph. The architecture can express the desire to ignore some events during pattern detection, and can also detect the absence of data as part of a pattern. The architecture facilitates efficient support for negation, ignorable events, and state cleanup based on predicate punctuations. | 11-17-2011 |
20110283295 | EXTENSIBILITY MODEL FOR STREAM-BASED OPERATORS AND AGGREGATES - An extensibility framework that allows a user to write user-defined modules that include user-defined operators (UDO) and user-defined aggregators (UDA) in a non-temporal fashion without the need to worry about temporal attributes of events (or event types). The temporal domain is managed on behalf of the user, and allows the user to write operators and aggregates in the temporal data streaming domain as well as to port existing libraries of non-temporal UDOs/UDAs to the temporal data streaming domain. Temporal attributes and event types are managed for non-temporal UDOs/UDAs by the extensibility framework on behalf of the UDO/UDA writer. Windows can be employed to bridge the gap between the non-temporal domains and temporal domains. Support for complex event processing (CEP) is provided in UDOs/UDAs for base classes related to a CEP operator, CEP aggregate, CEP time sensitive operator, and CEP time sensitive aggregate. | 11-17-2011 |
20110313844 | REAL-TIME-READY BEHAVIORAL TARGETING IN A LARGE-SCALE ADVERTISEMENT SYSTEM - A “Real-Time-Ready Analyzer” combines a data stream management system (DSMS) with a map-reduce (M-R) framework to construct a streaming map-reduce framework that is suitable for real-time Behavioral Targeting (BT) (or other temporal queries). The Real-Time-Ready Analyzer allows users to write “dual-intent” temporal analysis queries for BT. These queries are succinct and easy to express, scale well on large-scale offline data, and can also work over real-time data. Further, the Real-Time-Ready Analyzer uses the aforementioned streaming map-reduce framework to provide dual-intent algorithms for end-to-end BT phases. Experiments using real data from an advertisement system show that the Real-Time-Ready Analyzer is very efficient and incurs orders-of-magnitude lower development effort than conventional systems. | 12-22-2011 |
20120166417 | ASYNCHRONOUS TRANSFER OF STATE INFORMATION BETWEEN CONTINUOUS QUERY PLANS - Described herein are technologies pertaining to migrating state information of operators in a first continuous query plan to a second continuous query plan in an asynchronous manner, such that the first continuous query plan need not cease executing during the migrating of the state information. State information pertaining to stateful operators, such as join operators, is migrated from the first continuous query plan to the second continuous query plan by way of a transformation plan. State matching is utilized to generate the transformation plan. | 06-28-2012 |
20120297080 | Stream Operator - Aspects of the subject matter described herein relate to stream operators. In aspects, a stream event is received. Based on a timestamp associated with the stream event and the number of stream operators that have asked to be activated prior to or at the timestamp, zero or more stream objects are informed of one or more time events and torn down if no longer needed. In addition, a destination stream operator is instantiated, if needed, data (e.g., the timestamp and packet, if any) corresponding to the stream event is provided to a stream operator, output data, if any, is received from the stream operator, and the stream operator is torn down if it no longer needs state data between invocations. | 11-22-2012 |
20120304003 | OPERATOR STATE CHECKPOINTS - A checkpoint marker can be received at a first operator. The first operator can process the checkpoint marker by sending the checkpoint marker to a second operator and sending state checkpoint information representing a state of the first operator to a checkpoint writer. The checkpoint information can be used to rehydrate the state of one or more operators. For example, after a system failure, system shutdown, etc., checkpoint information can be received from a reader unit at a checkpoint information input queue of the first operator. A state of the first operator can be rehydrated using the checkpoint information. Processing of information in a data input queue of the first operator can be suspended while the checkpoint information is used to rehydrate the state of the first operator. Other operators in a system with the first operator (e.g., the second operator) may be checkpointed and rehydrated in the same manner as the first operator. | 11-29-2012 |
20120324453 | EFFICIENT LOGICAL MERGING OVER PHYSICALLY DIVERGENT STREAMS - A logical merge module is described herein for producing an output stream which is logically compatible with two or more physically divergent input streams. Representative applications of the logical merge module are also set forth herein. | 12-20-2012 |
20140250151 | DYNAMIC PATTERN MATCHING OVER ORDERED AND DISORDERED DATA STREAMS - Architecture introduces a new pattern operator referred to as called an augmented transition network (ATN), which is a streaming adaptation of non-reentrant, fixed-state ATNs for dynamic patterns. Additional user-defined information is associated with automaton states and is accessible to transitions during execution. ATNs are created that directly model complex pattern continuous queries with arbitrary cycles in a transition graph. The architecture can express the desire to ignore some events during pattern detection, and can also detect the absence of data as part of a pattern. The architecture facilitates efficient support for negation, ignorable events, and state cleanup based on predicate punctuations. | 09-04-2014 |