Glossary (StreamInsight)
The following terms are applicable to Microsoft StreamInsight.
Terms
| Term | Definition |
|---|---|
| adapter | (legacy term) Introduced in an earlier version of StreamInsight, a software transformer that delivers events into or out of a StreamInsight server. Adapters are still supported but have been superseded by the observable/observer source and sink model. See Other Terms: input adapter, output adapter. |
| adapter host | (legacy term) The root abstract class Adapter, which defines the handshake between the adapter and the StreamInsight server in the ENQUEUE interaction point. It provides all the required adapter services such as memory management, and exception handling. The adapter host invokes the methods implemented by the adapter writer, on behalf of the server. Note that the adapter host is transparent to the user. |
| bound stream | An event stream that contains all the information needed to produce events. Either the information is an already instantiated data source, or the information is sufficient for the StreamInsight server to start the data source. See Other Terms: event stream, unbound stream. |
| CEP engine | See Other Term: StreamInsight server |
| composable | Using query components (objects or operators) as reusable building blocks to form complex queries. This is done by linking query components together or encapsulating query components within each other. |
| complex event processing (CEP) | The continuous and incremental processing of event streams from multiple sources based on declarative query and pattern specifications with near-zero latency. The goal is to identify meaningful patterns, relationships, and data abstractions from seemingly unrelated events and to trigger immediate responses. |
| count window | Count windows have a variable window size that is defined by the count of distinct event start times along a timeline. Count windows move along the timeline with each distinct event start time. Hence, each new event that defines a new event start time will cause the creation of a new count window, as long as the count is fulfilled. See Other Terms: hopping window, sliding window, snapshot window, window. |
| CTI (current time increment) event | The event kind used to limit the impact of out-of-order events or event latency. The CTI event type consists of a single field that provides a current timestamp. The CTI event indicates to the StreamInsight server that no subsequent incoming insert events will have a start time less than the timestamp of the CTI event. In other words, the CTI is a special punctuation event that indicates the completeness of the existing events. By indicating completeness, the StreamInsight server can release the results of windowing or other aggregating operators that have accumulated state to the next downstream operator. See Other Terms: event kind, insert event. |
| dequeue | (legacy term) The operation used by the output adapter to pull one event, or a single batch of events, from the StreamInsight server to the output adapter. See Other Terms: output adapter, enqueue. |
| edge event | An edge event model represents an event occurrence whose payload is valid for a given interval of time, however, only the start time is known upon arrival to the StreamInsight server. The end time of the event is known later and updated. Examples of edge events are Windows processes, trace events from Event Tracing for Windows (ETW), a Web user session, or quantization of an analog signal. See Other Terms: event, event model, interval event model, time interval, point event model. |
| enqueue | (legacy term) The operation used by the input adapter to push one event, or a single batch of events, from memory to the StreamInsight server. See Other Terms: input adapter, dequeue. |
| event | The basic unit of data processed by the StreamInsight server. Each event contains a header that defines the event kind and the temporal properties of the event. An event (except the CTI event) typically contains an event payload, which is a .NET data structure that contains the data associated with the event. See Other Terms: event payload, point event model, edge event, interval event model. |
| event header | The portion of an event that defines the temporal properties of the event and the event kind. Temporal properties include a valid start time and end time associated with the event. See Other Terms: event, event kind, event payload. |
| event kind | Event metadata that defines the event type. StreamInsight supports two event kinds: Insert and CTI (current time increment). See Other Terms: insert event, CTI event. |
| event model | Event metadata that defines the temporal characteristics (shape) of the event. StreamInsight supports the following event models: point, interval and edge. See Other Terms: point event model, interval event model, edge event. |
| event payload | The data portion of an event in which the data fields are defined as CLR types. An event payload is a typed structure. See Other Terms: metadata header, event. |
| event shape | See Other Term: event model. |
| event stream | See Other Term: stream. |
| EventStreamGroup | A collection of streams that indicates that all branches in the streams are produced by a Group By operation. Any operation on an EventStreamGroup will operate against each stream in the group. The result of such an operation is a group of streams. |
| generic adapter | (legacy term) See Other Term: untyped adapter. |
| hopping window | A type of window in which consecutive windows "hop" forward in time by a fixed period. The window is defined by two time spans: the hop size H and the window size S. For every H time unit, a new window of size S is created. See Other Terms: count window, sliding window, snapshot window, window. |
| input adapter | (legacy term) A software transformer that accepts incoming event streams from external sources such as databases, files, ticker feeds, network ports, manufacturing devices and so on. The input adapter reads the incoming events in the format in which they are supplied and transforms this data into a format that is consumable by the StreamInsight server. Input adapters are still supported but have been generally superseded by the observable source model. See Other Terms: adapter, output adapter. |
| insert event | The event kind used to signify the arrival of an event into the stream. The insert event type consists of metadata that defines the valid lifetime of the event and the payload (data) fields of the event. See Other Terms: CTI event, event. |
| interval event model | The event model that represents an event whose payload is valid for a given period of time. The interval event model requires that both the start and end time of the interval be provided in the event metadata. Interval events are valid only for this specific interval. Examples include the width of an electronic pulse, the duration of (validity of) an auction bid, or a stock ticker activity in which the bid price for the stock is valid for a specific time interval. The start and end times defined by the temporal properties of the model determine the valid interval of the payload. See Other Terms: event, event model, point event model, edge event, time interval. |
| LINQ | Language Integrated Query. A set of extensions to the .NET Framework that encompass language-integrated query, set, and transform operations. See Other Terms: query template, standing query. |
| metadata header | See Other Term: event header. |
| operator | An atomic unit of a query as scheduled by the StreamInsight server to process the events on which the query is applied. Examples include SELECT, PROJECT, AGGREGATE, UNION, TOP K and JOIN. Operators are fully composable and have a specific number of inputs and outputs. See Other Terms: query, query template, composable. |
| output adapter | (legacy term) A software transformer that receives events processed by the StreamInsight server, transforms the events into a format expected by the output device, and emits the data to that device. The output device may be a database, text file, PDA, or other device. Output adapters are still supported but have been superseded by the observer sink model. See Other Terms: adapter, input adapter. |
| payload | See Other Term: event payload. |
| point event model | The event model that represents an event occurrence as of a single point in time. It is a subclass of the interval event model. The point event metadata provides only a valid start time. The StreamInsight server infers the valid end time by adding a tick (the smallest unit of time in the underlying time data type) to the start time to set the valid time interval for the event. Point events are valid only for this single instant of time. Examples of point events are the arrival of an email, a meter reading, a user Web click, a stock tick, or an entry into the Windows Event Log. See Other Terms: event, interval event model, time interval, edge event. |
| query | A LINQ query that runs continuously in the StreamInsight server processing events received from sources to which the query is bound and sending processed events to sinks to which it is bound. |
| query binding | The process of binding sources and sinks to a query . |
| query template | (legacy term) The fundamental unit of query composition in the adapter model. A query template defines the business logic required to continuously analyze and process events submitted to and emitted by the StreamInsight server. See Other Terms: query, query binding. |
| RankedEventStream | A stream of events that have been ranked. This stream source is required for the TopK operator and is not accepted as a stream source for any other operator. |
| retract event | An internal event kind used to modify an existing insert event by modifying the EndTime of the event. |
| sink | The structure or device that consumes events from the StreamInsight server. Examples include visualization tools, dashboards, files, tables, logs, and message queues. See Other Term: source. |
| sliding window | A window of fixed length L that moves along a timeline according to the stream’s events. With every event on the timeline, a new window is created, starting at the event’s start time. See Other Terms: hopping window, snapshot window, window. |
| snapshot window | Snapshot windows are defined according to the start and end times of the events in the stream, instead of a fixed grid along the timeline. The size and time period of the window is defined only by the events in the stream. For each pair of closest event endpoints (start time and end time), a snapshot window is created. By this definition, all event start and end times fall on window boundaries, never in between. That is, snapshot windows divide the timeline according to all occurring changes. See Other Terms: count window, hopping window, sliding window, window. |
| source | The structure or device that provides the source data to the StreamInsight server. Examples include files, Web logs, system logs, tables, message queues, market or news feeds, and sensors in manufacturing or logistics. See Other Term: sink. |
| standing query | A query that runs within the StreamInsight server performing continuous computation over the incoming events. As individual events arrive in the StreamInsight server, the events are processed by standing queries, which emit output events in response to the arrival of input events. Standing queries are written in the language framework LINQ. |
| stream | A collection of data that changes over time. All data in the StreamInsight server is organized into streams. A stream can be submitted from an external source to the StreamInsight server for processing and output to an external source. A stream can also be a queue of data events within a query that are exchanged from one operator to the next in the operator tree during query processing. See Other Term: temporal stream |
| stream consumer | The structure or device that consumes the output of a query. Examples are sink or another running query. See Other Terms: source, stream. |
| StreamInsight Event Flow Debugger | A stand-alone tool in the Microsoft StreamInsight platform that provides event-flow debugging and analysis. |
| StreamInsight LINQ | An implementation of LINQ used by the StreamInsight server to process and operate on event streams. StreamInsight LINQ expressions are defined over streams and yield streams. See Other Terms: LINQ, stream. |
| StreamInsight platform | A complete platform for the development of complex event processing applications. The platform consists of the StreamInsight server, Event Flow Debugging tool, Visual Studio IDE, and other components. |
| StreamInsight runtime | See Other Term: StreamInsight server |
| StreamInsight server | The core engine and framework components of Microsoft StreamInsight. The StreamInsight server can be used to process and analyze the event streams associated with a complex event processing application. |
| StreamInsight stream | See Other Term: temporal stream. |
| stream source | See Other Term: source. |
| temporal stream | A sequence of events that has a unique set of characteristics: each event in the stream has a timestamp, the stream contains CTI events, events in the stream observe the CTI model (that is, when a CTI appears in the temporal stream, no more events in the stream will have a timestamp earlier than the CTI timestamp). A temporal stream is implemented through the IStreamable<> interface and can be processed using StreamInsight LINQ. See Other Terms: CTI event, stream. |
| tick | The smallest unit of time in a datetime data type (nanoseconds). |
| time interval | A period of time in which a given event is valid. The valid time interval includes the valid start time, and all moments of time up to, but not including the valid end time. |
| typed adapter | (legacy term) An adapter that emits only a single event type. |
| unbound stream | An event stream that contains the definition of the event model or payload type, but does not define the data source. See Other Terms: stream, bound stream. |
| untyped adapter | (legacy term) An adapter that accepts or emits multiple event types in which the payload structure or the type of fields in the payload are not known in advance. Examples are events from a CSV or text file, a SQL table, or a socket. See Other Term: typed adapter. |
| window | A subset of events within a stream that fall within some period of time; that is, a window contains event data along a timeline. Windows are required to enable window-based operators such as sum, avg, and count, but can also be combined with other data processing operations. See Other Terms: count window, hopping window, sliding window, snapshot window. |
| WindowStream | A stream of windows in which each window contains a collection of events. All events in the stream share the common property of falling into one or more windows. This could be a time-based window whose start and end times can be accessed. |