baseplate.events
¶
Building Events¶
Thrift Schema v2 Events¶
For modern Thrift-based events: import the event schemas into your project, instantiate and fill out an event object, and pass it into the queue:
import time
import uuid
from baseplate.events import EventQueue, serialize_v2_event
from event_schemas.event.ttypes import Event
def make_wsgi_app(app_config):
...
queue = EventQueue("v2", event_serializer=serialize_v2_event)
baseplate.add_to_context("events_v2", queue)
...
def my_handler(request):
event = Event(
source="baseplate",
action="test",
noun="baseplate",
client_timestamp=time.time() * 1000,
uuid=str(uuid.uuid4()),
)
request.events_v2.put(ev2)
Legacy schemaless events¶
For legacy schemaless events, you can use these helper objects to build payloads:
-
class
baseplate.events.
FieldKind
¶ Field kinds.
-
NORMAL
= None¶ For fields normal fields with no hashing/indexing requirements.
-
OBFUSCATED
= 'obfuscated_data'¶ For fields containing sensitive information like IP addresses that must be treated with care.
-
HIGH_CARDINALITY
= 'interana_excluded'¶ For fields that should not be indexed due to high cardinality (e.g. not used in Interana)
-
-
class
baseplate.events.
Event
(topic, event_type, timestamp=None, id=None)¶ An event.
-
get_field
(key)¶ Get the value of a field in the event.
If the field is not present,
None
is returned.Parameters: key (str) – The name of the field.
-
set_field
(key, value, obfuscate=False, kind=<FieldKind.NORMAL: None>)¶ Set the value for a field in the event.
Parameters: - key (str) – The name of the field.
- value – The value to set the field to. Should be JSON serializable.
- kind (baseplate.events.FieldKind) – The kind the field is. Used to determine what section of the payload the field belongs in when serialized.
-
Queing Events¶
-
class
baseplate.events.
EventQueue
(name, event_serializer=<function serialize_v1_event>)¶ A queue to transfer events to the publisher.
Parameters: - name (str) – The name of the event queue to send to. This specifies which publisher should send the events which can be useful for routing to different event pipelines (prod/test/v2 etc.).
- event_serializer (callable) – A callable that takes an event object and returns serialized bytes ready to send on the wire. See below for options.
-
put
(event)¶ Add an event to the queue.
The queue is local to the server this code is run on. The event publisher on the server will take these events and send them to the collector.
Parameters: event – The event to send. The type of event object passed in depends on the selected event_serializer
.Raises: EventTooLargeError
The serialized event is too large.Raises: EventQueueFullError
The queue is full. Events are not being published fast enough.
The EventQueue
also implements
ContextFactory
so it can be used with
add_to_context()
:
event_queue = EventQueue("production")
baseplate.add_to_context("events_production", event_queue)
It can then be used from the context object during requests:
def some_service_method(self, context):
event = Event(...)
context.events_production.put(event)
Serializers¶
The event_serializer
parameter to EventQueue
is a callable
which serializes a given event object. The default is the original schemaless
format. This can be overridden by passing in a different serializer. Baseplate
comes with a serializer for the new Thrift schema based V2 event system as
well:
-
baseplate.events.
serialize_v1_event
(event)¶ Serialize an Event object for the V1 event protocol.
Parameters: event (baseplate.events.Event) – An event object.
-
baseplate.events.
serialize_v2_event
(event)¶ Serialize a Thrift struct to bytes for the V2 event protocol.
Parameters: event – A Thrift struct from the event schemas.
Exceptions¶
-
exception
baseplate.events.
EventError
¶ Base class for event related exceptions.
-
exception
baseplate.events.
EventTooLargeError
(size)¶ Raised when a serialized event is too large to send.
-
exception
baseplate.events.
EventQueueFullError
¶ Raised when the queue of events is full.
This usually indicates that the event publisher is having trouble talking to the event collector.
Publishing Events¶
Events that are put onto an EventQueue
are consumed by a separate
process and published to the remote event collector service. The publisher is
in baseplate and can be run as follows:
python -m baseplate.events.publisher --queue-name something config_file.ini
The publisher will look at the specified INI file to find its configuration.
Given a queue name of something
(as in the example above), it will expect a
section in the INI file called [event-publisher:something]
with content
like below:
[event-publisher:something]
collector.hostname = some-domain.example.com
key.name = NameOfASecretKey
key.secret = Base64-encoded-blob-of-randomness
metrics.namespace = a.name.to.put.metrics.under
metrics.endpoint = the-statsd-host:1234