Creating Event Types

This topic provides guidelines for creating event types. An event type defines the events that are published by the event source or consumed by the event sink. In the development experience, they are specified either as single primitive types, as .NET Framework classes, or structs and define the data (payload) that is associated with each event in the event stream. Event types are used in the following stages of development:

1. Writing a typed adapter. A typed adapter uses the event type as a generic parameter so that the proper structure is initialized when creating new event objects to be enqueued.

2. Writing a query template. A query template is specified in LINQ on top of a CepStream object, whose definition is based on an event type.

For more information about events and event streams, see StreamInsight Server Concepts.

Determine the Event Structure

When the event payload (that is, the number of fields in the payload and their types) is fixed and known in advance, you can create a typed adapter. With a typed adapter, all instances of the adapter produce the same fixed payload format. This is typically the case for event sources with a fixed data structure.

When you create a typed adapter, you define a .NET class or struct that represents the fixed payload, or use a primitive type if your event payload can be represented by a single field. When using structs or classes, you can use only public fields and properties as payload fields. Private fields and properties, and class methods are ignored and cannot be used in the event type. You use this event type declaration to populate the event type in the input adapter and to obtain the result from the typed event type in the output adapter. The following example defines a simple event type that has two payload fields, V1 and V2, of type int.

    public class MyPayload
    {
        public int V1 { get; set; }
        public int V2 { get; set; }
    }

Another example shows how to use a nested event type:

    public class ComplexPayload
    {
        public ValueType Value { get; set; }
        public bool Status  { get; set; }
    }

    public class ValueType
    {
        public double Value { get; set; }
        public int Quality  { get; set; }
    }

An untyped adapter is useful when you want the same adapter for a specific source or sink to be configurable so that it can handle multiple event types. The event payload format for these event types is provided to the adapter as part of a configuration specification when the adapter is bound to the query. An example of such a scenario is a .csv file that contains a varying number of fields. The exact set of fields that make up a payload might be determined by the query designer (and not the developer who is building the adapter) only at the time of binding and instantiating the query. Another example is an adapter for SQL tables, where the type of the events that are produced depends on the schema of the table or the query issued against the table. For more information, see Creating Input and Output Adapters.

The specification of a query template is based on a CepStream object with a specific event type, that has to be known at design time. Intermediate CepStream objects along the query template specification can contain new types that are implicitly defined in project clauses ("select" in LINQ), using member init expressions.

Payload Field Requirements

When creating event types, consider the following payload field requirements and functionality.

• An event structure cannot have an empty payload structure. At least one field is required.

• Both scalar and elementary .NET Framework types and nested types can be used for the payload fields. See the section 'Supported Data Types' that follows.

• Fields cannot be modified by using custom attributes.

• Event types in the StreamInsight server are an ordered list of fields instead of .NET structs, which do not impose an order on its fields. The order of event fields becomes important for untyped adapters. These adapters access fields by ordinal because the fields are not known at adapter design time. As a default convention, a .NET struct, which is used as an event type, orders its fields according to the lexicographic order of their names. 

• Nullability of the field is inferred. For example, int? will be nullable, but int will not be nullable. String and byte[] types are always nullable.

• The default size of byte[] is 512.

• The maximum length of string fields is only bound by the page size for the entire event, which is 16k bytes.

Event Size

There is no explicit limitation on the number of fields that can be defined in the event type. The number of fields depends on the type of the individual fields, their size, and nullability.

The event page size in the StreamInsight server is 16K. Because an event cannot span across multiple pages, 16K minus some overhead is the effective maximum event size (including payload and timestamp fields). In addition to the fixed overhead incurred by the page header, event header, and system fields such as timestamp, nullability adds to the variable overhead in the order of N/8 adjusted to an upper boundary.

To maximize event page utilization, we recommend the following guidelines:

• Avoid nullable fields.

• Minimize the use of string and byte[] fields.

• Keep the event lifetimes only as long as required by the semantics of the respective scenario, so that memory needed for event state can be released more efficiently in the engine

Discoverability of Event Types

Once an event type has been created, the next step is to ensure its discoverability by dependent projects. Note that the .NET environment provides developers with the ability to build modules in a loosely coupled, distributed manner and ship them as assemblies. These assemblies can then be put together to build an integrated application, tested, and deployed to production.

Recall that a runnable StreamInsight query is the result of successfully binding a query template with instances of input and output adapters, based on matching event types at the input and output of the query. Depending on the development environment, you may need to provide access to the event type to others. For example, the adapter developer and the query developer may have to work on their respective modules independently. Consider the following scenarios for event type discoverability:

• If the adapter developer is the publisher of the event type, the adapter developer can write the adapter using the StreamInsight managed API and ship the .NET assembly. Other developers can reflect on the .NET assembly by using ildasm.exe or by referencing the DLL from inside the Visual Studio project and determine the event type. Using this method, access to the StreamInsight server is not required to access the event type.

• If the StreamInsight server administrator is the publisher of the event type, the administrator or developer can deploy the event type in the StreamInsight server. A query developer who has access to the StreamInsight server can then use the applicable object model API calls to inspect metadata items and determine the event type. The adapter developer can also write a typed adapter that supports this event type.

• If the adapter developer and the query developer are actually disconnected -(each working with his or her own representation of a common event type), the validation of a match between their respective event types will occur at query bind time. The bind will be successful if there is a match.

• If the query developer has no access to the adapter assembly or the StreamInsight server in which the adapter is registered, the developer can determine the event type by referring to the product literature for the adapter, or through common knowledge of data transfer protocols specific to that domain (for example, Financial Information Exchange (FIX) or Reuters Market Data System (RMDS) in financial trading).

Supported Data Types

In StreamInsight, each event field and expression has a specific data type. StreamInsight supports the following data types. Event payloads can also contain nested types that are composed of these data types.

1 Does not include nullable type.

During adapter development, event timestamps are specified using the DataTimeOffset type. However, only the DateTime type is supported in the definition of the event type payload fields. In scenarios in which you import data of type DateTimeOffset into an event field, you must convert the data to DateTime of kind UTC. For example, a payload field may be filled in from a SQL Server database, or by copying the event timestamp fields into payload fields for computation.

See Also

Concepts

Creating Input and Output Adapters

StreamInsight Server Concepts

StreamInsight End-to-End Example