baseplate.events

Building Events

Thrift Schema v2 Events

For modern Thrift-based events: import the event schemas into your project, instantiate and fill out an event object, and pass it into the queue:

import time
import uuid

from baseplate.events import EventQueue, serialize_v2_event

from event_schemas.event.ttypes import Event


def make_wsgi_app(app_config):
   ...

   queue = EventQueue("v2", event_serializer=serialize_v2_event)
   baseplate.add_to_context("events_v2", queue)

   ...


def my_handler(request):
   event = Event(
      source="baseplate",
      action="test",
      noun="baseplate",
      client_timestamp=time.time() * 1000,
      uuid=str(uuid.uuid4()),
   )
   request.events_v2.put(ev2)

Legacy schemaless events

For legacy schemaless events, you can use these helper objects to build payloads:

class baseplate.events.FieldKind

Field kinds.

NORMAL = None

For fields normal fields with no hashing/indexing requirements.

OBFUSCATED = 'obfuscated_data'

For fields containing sensitive information like IP addresses that must be treated with care.

HIGH_CARDINALITY = 'interana_excluded'

For fields that should not be indexed due to high cardinality (e.g. not used in Interana)

class baseplate.events.Event(topic, event_type, timestamp=None, id=None)

An event.

get_field(key)

Get the value of a field in the event.

If the field is not present, None is returned.

Parameters:key (str) – The name of the field.
set_field(key, value, obfuscate=False, kind=<FieldKind.NORMAL: None>)

Set the value for a field in the event.

Parameters:
  • key (str) – The name of the field.
  • value – The value to set the field to. Should be JSON serializable.
  • kind (baseplate.events.FieldKind) – The kind the field is. Used to determine what section of the payload the field belongs in when serialized.

Queing Events

class baseplate.events.EventQueue(name, event_serializer=<function serialize_v1_event>)

A queue to transfer events to the publisher.

Parameters:
  • name (str) – The name of the event queue to send to. This specifies which publisher should send the events which can be useful for routing to different event pipelines (prod/test/v2 etc.).
  • event_serializer (callable) – A callable that takes an event object and returns serialized bytes ready to send on the wire. See below for options.
put(event)

Add an event to the queue.

The queue is local to the server this code is run on. The event publisher on the server will take these events and send them to the collector.

Parameters:event – The event to send. The type of event object passed in depends on the selected event_serializer.
Raises:EventTooLargeError The serialized event is too large.
Raises:EventQueueFullError The queue is full. Events are not being published fast enough.

The EventQueue also implements ContextFactory so it can be used with add_to_context():

event_queue = EventQueue("production")
baseplate.add_to_context("events_production", event_queue)

It can then be used from the context object during requests:

def some_service_method(self, context):
    event = Event(...)
    context.events_production.put(event)

Serializers

The event_serializer parameter to EventQueue is a callable which serializes a given event object. The default is the original schemaless format. This can be overridden by passing in a different serializer. Baseplate comes with a serializer for the new Thrift schema based V2 event system as well:

baseplate.events.serialize_v1_event(event)

Serialize an Event object for the V1 event protocol.

Parameters:event (baseplate.events.Event) – An event object.
baseplate.events.serialize_v2_event(event)

Serialize a Thrift struct to bytes for the V2 event protocol.

Parameters:event – A Thrift struct from the event schemas.

Exceptions

exception baseplate.events.EventError

Base class for event related exceptions.

exception baseplate.events.EventTooLargeError(size)

Raised when a serialized event is too large to send.

exception baseplate.events.EventQueueFullError

Raised when the queue of events is full.

This usually indicates that the event publisher is having trouble talking to the event collector.

Publishing Events

Events that are put onto an EventQueue are consumed by a separate process and published to the remote event collector service. The publisher is in baseplate and can be run as follows:

python -m baseplate.events.publisher --queue-name something config_file.ini

The publisher will look at the specified INI file to find its configuration. Given a queue name of something (as in the example above), it will expect a section in the INI file called [event-publisher:something] with content like below:

[event-publisher:something]
collector.hostname = some-domain.example.com

key.name = NameOfASecretKey
key.secret = Base64-encoded-blob-of-randomness

metrics.namespace = a.name.to.put.metrics.under
metrics.endpoint = the-statsd-host:1234