Glossary (StreamInsight)
The following terms are applicable to Microsoft StreamInsight.
Terms
Term |
Definition |
|---|---|
adapter |
A software transformer that delivers events into or out of a StreamInsight server. See Other Terms: input adapter, output adapter. |
adapter host |
The root abstract class Adapter, which defines the handshake between the adapter and the StreamInsight server in the ENQUEUE interaction point. It provides all the required adapter services such as memory management, and exception handling. The adapter host invokes the methods implemented by the adapter writer, on behalf of the server. Note that the adapter host is transparent to the user. |
bound stream |
An event stream that contains all the information needed to produce events. Either the information is an already instantiated data source, or the information is sufficient for the StreamInsight server to start the data source. See Other Terms: event stream, unbound stream. |
CEP engine |
See Other Term: StreamInsight server |
StreamInsight LINQ |
An implementation of LINQ used by the StreamInsight server to process and operate on event streams. StreamInsight LINQ expressions are defined over streams and yield streams. See Other Terms: LINQ, streams. |
composable |
Using query components (objects or operators) as reusable building blocks to form complex queries. This is done by linking query components together or encapsulating query components within each other. |
complex event processing (CEP) |
The continuous and incremental processing of event streams from multiple sources based on declarative query and pattern specifications with near-zero latency. The goal is to identify meaningful patterns, relationships, and data abstractions from seemingly unrelated events and to trigger immediate responses. |
count window |
Count windows have a variable window size that is defined by the count of distinct event start times along a timeline. Count windows move along the timeline with each distinct event start time. Hence, each new event that defines a new event start time will cause the creation of a new count window, as long as the count is fulfilled. See Other Terms: hopping window, sliding window, snapshot window, window. |
CTI (current time increment) event |
The event kind used to limit the impact of out-of-order events or event latency. The CTI event type consists of a single field that provides a current timestamp. The CTI event indicates to the StreamInsight server that no subsequent incoming insert events will have a start time less than the timestamp of the CTI event. In other words, the CTI is a special punctuation event that indicates the completeness of the existing events. By indicating completeness, the StreamInsight server can release the results of windowing or other aggregating operators that have accumulated state to the next downstream operator. See Other Terms: event kind, insert event. |
dequeue |
The operation used by the output adapter to pull one event, or a single batch of events, from the StreamInsight server to the output adapter. See Other Terms: output adapter, enqueue. |
edge event |
An edge event model represents an event occurrence whose payload is valid for a given interval of time, however, only the start time is known upon arrival to the StreamInsight server. The end time of the event is known later and updated. Examples of edge events are Windows processes, trace events from Event Tracing for Windows (ETW), a Web user session, or quantization of an analog signal. See Other Terms: event, event model, interval event, valid time interval, and point event. |
enqueue |
The operation used by the input adapter to push one event, or a single batch of events, from memory to the StreamInsight server. See Other Terms: input adapter, dequeue. |
event |
The basic unit of data processed by the StreamInsight server. Each event contains a header that defines the event kind and the temporal properties of the event. An event (except the CTI event) typically contains an event payload, which is a .NET data structure that contains the data associated with the event. See Other Terms: event payload, point event, edge event, interval event. |
event header |
The portion of an event that defines the temporal properties of the event and the event kind. Temporal properties include a valid start time and end time associated with the event. See Other Terms: event, event kind, and event payload. |
event kind |
Event metadata that defines the event type. StreamInsight supports two event kinds: Insert and CTI (current time increment). See Other Terms: insert event, CTI event. |
event model |
Event metadata that defines the temporal characteristics (shape) of the event. StreamInsight supports the following event models: point, interval and edge. See Other Terms: point event, interval event and edge event. |
event payload |
The data portion of an event in which the data fields are defined as CLR types. An event payload is a typed structure. See Other Term: metadata header, event. |
event shape |
See Other Term: event model. |
event stream |
See Other Term: stream. |
EventStreamGroup |
A collection of streams that indicates that all branches in the streams are produced by a Group By operation. Any operation on an EventStreamGroup will operate against each stream in the group. The result of such an operation is a group of streams. |
generic adapter |
See Other Term: untyped adapter. |
hopping window |
A type of window in which consecutive windows "hop" forward in time by a fixed period. The window is defined by two time spans: the hop size H and the window size S. For every H time unit, a new window of size S is created. See Other Terms: count window, sliding window, snapshot window, window. |
input adapter |
A software transformer that accepts incoming event streams from external sources such as databases, files, ticker feeds, network ports, manufacturing devices and so on. The input adapter reads the incoming events in the format in which they are supplied and transforms this data into a format that is consumable by the StreamInsight server. See Other Terms: adapter, output adapter. |
insert event |
The event kind used to signify the arrival of an event into the stream. The insert event type consists of metadata that defines the valid lifetime of the event and the payload (data) fields of the event. See Other Terms: CTI event, event. |
interval event model |
The event model that represents an event whose payload is valid for a given period of time. The interval event model requires that both the start and end time of the interval be provided in the event metadata. Interval events are valid only for this specific interval. Examples include the width of an electronic pulse, the duration of (validity of) an auction bid, or a stock ticker activity in which the bid price for the stock is valid for a specific time interval. The start and end times defined by the temporal properties of the model determine the valid interval of the payload. See Other Terms: event, event model, point event, edge event and valid time interval. |
LINQ |
Language Integrated Query. A set of extensions to the .NET Framework that encompass language-integrated query, set, and transform operations. See Other Terms: query template, standing query. |
metadata header |
See Other Term: event metadata. |
operator |
An atomic unit of a query as scheduled by the StreamInsight server to process the events on which the query is applied. Examples include SELECT, PROJECT, AGGREGATE, UNION, TOP K and JOIN. Operators are fully composable and have a specific number of inputs and outputs. See Other Terms: query, query template, composable. |
output adapter |
A software transformer that receives events processed by the StreamInsight server, transforms the events into a format expected by the output device, and emits the data to that device. The output device may be a database, text file, PDA, or other device. See Other Terms: adapter and input adapter. |
payload |
See Other Term: event payload. |
point event model |
The event model that represents an event occurrence as of a single point in time. It is a subclass of the interval event model. The point event metadata provides only a valid start time. The StreamInsight server infers the valid end time by adding a tick (the smallest unit of time in the underlying time data type) to the start time to set the valid time interval for the event. Point events are valid only for this single instant of time. Examples of point events are the arrival of an email, a meter reading, a user Web click, a stock tick, or an entry into the Windows Event Log. See Other Terms: event, interval event, valid time interval, edge event. |
query |
An instance of a query template that runs continuously in the StreamInsight server processing events received from instances of input adapters to which the query is bound and sending processed events to instances of output adapters to which it is bound. |
query binding |
The process of binding instances of input adapters and instances of output adapters to an instance of a query template. |
query template |
The fundamental unit of query composition. A query template defines the business logic required to continuously analyze and process events submitted to and emitted by the StreamInsight server. See Other Terms: query, query binding. |
RankedEventStream |
A stream of events that have been ranked. This stream source is required for the TopK operator and is not accepted as a stream source for any other operator. |
retract event |
An internal event kind used to modify an existing insert event by modifying the EndTime of the event. |
sink |
The structure or device that consumes events from the StreamInsight server via the output adapter. Examples include visualization tools, dashboards, files, tables, logs, and message queues. See Other Term: source. |
sliding window |
A window of fixed length L that moves along a timeline according to the stream’s events. With every event on the timeline, a new window is created, starting at the event’s start time. See Other Terms: hopping window, snapshot window, window. |
snapshot window |
Snapshot windows are defined according to the start and end times of the events in the stream, instead of a fixed grid along the timeline. The size and time period of the window is defined only by the events in the stream. For each pair of closest event endpoints (start time and end time), a snapshot window is created. By this definition, all event start and end times fall on window boundaries, never in between. That is, snapshot windows divide the timeline according to all occurring changes. See Other Terms: count window, hopping window, sliding window, window. |
source |
The structure or device that provides the source data to the StreamInsight server via the input adapter. Examples include files, Web logs, system logs, tables, message queues, market or news feeds, and sensors in manufacturing or logistics. See Other Term: sink. |
standing query |
An instantiation of a query template that runs within the StreamInsight server performing continuous computation over the incoming events. As individual events arrive in the StreamInsight server, the events are processed by standing queries, which emit output events in response to the arrival of input events. Standing queries are written in the language framework LINQ. |
stream |
A collection of data that changes over time. All data in the StreamInsight server is organized into streams. A stream can be submitted from an external source to the StreamInsight server for processing and output to an external source. A stream can also be a queue of data events within a query that are exchanged from one operator to the next in the operator tree during query processing. |
stream consumer |
The structure or device that consumes the output of a query. Examples are an output adapter or another running query. See Other Terms: stream source, stream. |
StreamInsight Event Flow Debugger |
A stand-alone tool in the Microsoft StreamInsight platform that provides event-flow debugging and analysis. |
StreamInsight platform |
A complete platform for the development of complex event processing applications. The platform consists of the StreamInsight server, Event Flow Debugging tool, Visual Studio IDE, and other components. |
StreamInsight runtime |
See Other Term: StreamInsight server |
StreamInsight server |
The core engine and adapter framework components of Microsoft StreamInsight. The StreamInsight server can be used to process and analyze the event streams associated with a complex event processing application. |
StreamInsight stream |
See Other Term: event stream. |
stream source |
See Other Terms: source. |
tick |
The smallest unit of time in a datetime data type (nanoseconds). |
time interval |
A period of time in which a given event is valid. The valid time interval includes the valid start time, and all moments of time up to, but not including the valid end time. |
typed adapter |
An adapter that emits only a single event type. See Other Term: typed event. |
unbound stream |
An event stream that contains the definition of the event model or payload type, but does not define the data source. See Other Terms: event stream, bound stream. |
untyped adapter |
An adapter that accepts or emits multiple event types in which the payload structure or the type of fields in the payload are not known in advance. Examples are events from a CSV or text file, a SQL table, or a socket. See Other Terms: typed adapter. |
window |
A subset of events within a stream that fall within some period of time; that is, a window contains event data along a timeline. Windows are required to enable window-based operators such as sum, avg, and count, but can also be combined with other data processing operations. See Other Terms: count window, hopping window, sliding window, snapshot window. |
WindowStream |
A stream of windows in which each window contains a collection of events. All events in the stream share the common property of falling into one or more windows. This could be a time-based window whose start and end times can be accessed. |