baseplate.core

The heart of the Baseplate framework is its diagnostics system. Here’s an incomplete example of an application built with the framework:

def do_something(request):
    request.some_redis_client.ping()

def make_app(app_config):
    ... snip ...

    baseplate = Baseplate()
    baseplate.configure_metrics(metrics_client)
    baseplate.add_to_context(
        "some_redis_client", RedisContextFactory(redis_pool))

    ... snip ...

When a request is made which routes to the do_something handler, a ServerSpan is automatically created to represent the time spent processing the request in our application. If the incoming request has trace headers, the constructed server span will have the same IDs as the upstream service’s child span.

When we call request.some_redis_client.ping() in the handler, Baseplate will create a child Span object to represent the time taken talking to redis.

The creation of the server and child spans will trigger updates on all the ServerSpanObserver and SpanObserver objects registered. Because we called baseplate.configure_metrics in our setup, this means we have observers that send statsd metrics so Baseplate will automatically send metrics on how long it took our application to do_something and how long Redis took to respond to our ping to statsd/Graphite without any extra code in our application.

Note

The documentation below explains how all this works under the hood. If you just want to write an application, you can skip on to how to integrate Baseplate with your application framework or how to use client libraries with diagnostic instrumentation.

Baseplate

At the root of each application is a single instance of Baseplate. This object can be integrated with various other frameworks (e.g. Thrift, Pyramid, etc.) using one of the integrations.

class baseplate.core.Baseplate

The core of the Baseplate diagnostics framework.

This class coordinates monitoring and tracing of service calls made to and from this service. See baseplate.integration for how to integrate it with the application framework you are using.

register(observer)

Register an observer.

Parameters:observer (baseplate.core.BaseplateObserver) – An observer.
configure_logging()

Add request context to the logging system.

configure_metrics(metrics_client)

Send timing metrics to the given client.

This also adds a baseplate.metrics.Batch object to the metrics attribute on the context object where you can add your own application-specific metrics. The batch is automatically flushed at the end of the request.

Parameters:metrics_client (baseplate.metrics.Client) – Metrics client to send request metrics to.
configure_tracing(tracing_client, *args, **kwargs)

Collect and send span information for request tracing.

When configured, this will send tracing information automatically collected by Baseplate to the configured distributed tracing service.

Parameters:tracing_client (baseplate.diagnostics.tracing.TracingClient) – Tracing client to send request traces to.
configure_error_reporting(client)

Send reports for unexpected exceptions to the given client.

This also adds a raven.Client object to the sentry attribute on the context object where you can send your own application-specific events.

Parameters:client (raven.Client) – A configured raven client.
add_to_context(name, context_factory)

Add an attribute to each request’s context object.

On each request, the factory will be asked to create an appropriate object to attach to the context object.

Parameters:
  • name (str) – The attribute on the context object to attach the created object to. This may also be used for metric/tracing purposes so it should be descriptive.
  • context_factory (baseplate.context.ContextFactory) – A factory.
make_server_span(context, name, trace_info=None)

Return a server span representing the request we are handling.

In a server, a server span represents the time spent on a single incoming request. Any calls made to downstream services will be new child spans of the server span, and the server span will in turn be the child span of whatever upstream request it is part of, if any.

Parameters:
  • context – The context object for this request.
  • name (str) – A name to identify the type of this request, e.g. a route or RPC method name.
  • trace_info (baseplate.core.TraceInfo) – The trace context of this request as passed in from upstream. If None, a new trace context will be generated.
class baseplate.core.TraceInfo

Trace context for a span.

If this request was made at the behest of an upstream service, the upstream service should have passed along trace information. This class is used for collecting the trace context and passing it along to the server span.

classmethod from_upstream(trace_id, parent_id, span_id, sampled, flags)

Build a TraceInfo from individual headers.

Parameters:
  • trace_id (int) – The ID of the trace.
  • parent_id (int) – The ID of the parent span.
  • span_id (int) – The ID of this span within the tree.
  • sampled (bool) – Boolean flag to determine request sampling.
  • flags (int) – Bit flags for communicating feature flags downstream
Raises:

ValueError if any of the values are inappropriate.

Convenience Methods

Baseplate comes with some core monitoring observers built in and just requires you to configure them. You can enable them by calling the relevant methods on your application’s Baseplate object.

Additionally, Baseplate provides helpers which can be attached to the context object in requests. These helpers make the passing of trace information and collection of spans automatic and transparent. Because this pattern is so common, Baseplate has a special kind of observer for it which can be registered with add_to_context(). See the baseplate.context package for a list of helpers included.

Spans

Each time a new request comes in to be served, the time taken to handle the request is represented by a new ServerSpan instance. During the course of handling that request, our application might make calls to remote services or do expensive calculations, the time spent can be represented by child Span instances.

Spans have names and IDs and track their parent relationships. When calls are made to remote services, the information that identifies the local child span representing that service call is passed along to the remote service and becomes the server span in the remote service. This allows requests to be traced across the infrastructure.

Small bits of data, called annotations, can be attached to spans as well. This could be the URL fetched, or how many items were sent in a batch, or whatever else might be helpful.

class baseplate.core.ServerSpan(trace_id, parent_id, span_id, sampled, flags, name, context)

A server span represents a request this server is handling.

The server span is available on the context object during requests as the trace attribute.

finish(exc_info=None)

Record the end of the span.

Parameters:exc_info – If the span ended because of an exception, this is the exception information. The default is None which indicates normal exit.
log(name, payload=None)

Add a log entry to the span.

Log entries are timestamped events recording notable moments in the lifetime of a span.

Parameters:
  • name (str) – The name of the log entry. This should be a stable identifier that can apply to multiple span instances.
  • payload – Optional log entry payload. This can be arbitrary data.
make_child(name, local=False, component_name=None)

Return a child Span whose parent is this Span.

The child span can either be a local span representing an in-request operation or a span representing an outbound service call.

In a server, a local span represents the time spent within a local component performing an operation or set of operations. The local component is some grouping of business logic, which is then split up into operations which could each be wrapped in local spans.

Parameters:
  • name (str) – Name to identify the operation this span is recording.
  • local (bool) – Make this span a LocalSpan if True, otherwise make this span a base Span.
  • component_name (str) – Name to identify local component this span is recording in if it is a local span.
register(observer)

Register an observer to receive events from this span.

set_tag(key, value)

Set a tag on the span.

Tags are arbitrary key/value pairs that add context and meaning to the span, such as a hostname or query string. Observers may interpret or ignore tags as they desire.

Parameters:
  • key (str) – The name of the tag.
  • value – The value of the tag, must be a string/boolean/number.
start()

Record the start of the span.

This notifies any observers that the span has started, which indicates that timers etc. should start ticking.

Spans also support the context manager protocol, for use with Python’s with statement. When the context is entered, the span calls start() and when the context is exited it automatically calls finish().

class baseplate.core.Span(trace_id, parent_id, span_id, sampled, flags, name, context)

A span represents a single RPC within a system.

register(observer)

Register an observer to receive events from this span.

start()

Record the start of the span.

This notifies any observers that the span has started, which indicates that timers etc. should start ticking.

Spans also support the context manager protocol, for use with Python’s with statement. When the context is entered, the span calls start() and when the context is exited it automatically calls finish().

set_tag(key, value)

Set a tag on the span.

Tags are arbitrary key/value pairs that add context and meaning to the span, such as a hostname or query string. Observers may interpret or ignore tags as they desire.

Parameters:
  • key (str) – The name of the tag.
  • value – The value of the tag, must be a string/boolean/number.
log(name, payload=None)

Add a log entry to the span.

Log entries are timestamped events recording notable moments in the lifetime of a span.

Parameters:
  • name (str) – The name of the log entry. This should be a stable identifier that can apply to multiple span instances.
  • payload – Optional log entry payload. This can be arbitrary data.
finish(exc_info=None)

Record the end of the span.

Parameters:exc_info – If the span ended because of an exception, this is the exception information. The default is None which indicates normal exit.
make_child(name, local=False, component_name=None)

Return a child Span whose parent is this Span.

Observers

To actually do something with all these spans, Baseplate provides observer interfaces which receive notification of events happening in the application via calls to various methods.

The base type of observer is BaseplateObserver which can be registered with the root Baseplate instance using the register() method. Whenever a new server span is created in your application (i.e. a new request comes in to be served) the observer has its on_server_span_created() method called with the relevant details. This method can register ServerSpanObserver instances with the new server span to receive events as they happen.

Spans can be notified of five common events:

New child spans are created in the application automatically by various client library instrumentations e.g. for a call to a remote service or database, and can also be created explicitly for local actions like expensive computations. The handler can register new SpanObserver instances with the new child span to receive events as they happen.

It’s up to the observers to attach meaning to these events. For example, the metrics observer would start a timer on_start() and record the elapsed time to statsd on_finish().

class baseplate.core.BaseplateObserver

Interface for an observer that watches Baseplate.

on_server_span_created(context, server_span)

Called when a server span is created.

Baseplate calls this when a new request begins.

Parameters:
class baseplate.core.ServerSpanObserver

Interface for an observer that watches the server span.

on_child_span_created(span)

Called when a child span is created.

SpanObserver objects call this when a new child span is created.

Parameters:span (baseplate.core.Span) – The new child span.
on_finish(exc_info)

Called when the observed span is finished.

Parameters:exc_info – If the span ended because of an exception, the exception info. Otherwise, None.
on_log(name, payload)

Called when a log entry is added to the span.

on_set_tag(key, value)

Called when a tag is set on the observed span.

on_start()

Called when the observed span is started.

class baseplate.core.SpanObserver

Interface for an observer that watches a span.

on_start()

Called when the observed span is started.

on_set_tag(key, value)

Called when a tag is set on the observed span.

on_log(name, payload)

Called when a log entry is added to the span.

on_finish(exc_info)

Called when the observed span is finished.

Parameters:exc_info – If the span ended because of an exception, the exception info. Otherwise, None.
on_child_span_created(span)

Called when a child span is created.

SpanObserver objects call this when a new child span is created.

Parameters:span (baseplate.core.Span) – The new child span.

Edge Request Context

The EdgeRequestContext provides an interface into both authentication and context information about the original request from a user. For edge services, it provides helpers to create the initial object and serialize the context information into the appropriate headers. Once this object is created and attached to the context, Baseplate will automatically forward the headers to downstream services so they can access the authentication and context data as well.

class baseplate.core.EdgeRequestContextFactory(secrets)

Factory for creating EdgeRequestContext objects.

Every application should set one of these up. Edge services that talk directly with clients should use new() directly. For internal services, pass the object off to Baseplate’s framework integration (Thrift/Pyramid) for automatic use.

Parameters:secrets (baseplate.secrets.SecretsStore) – A configured secrets store.
new(authentication_token=None, loid_id=None, loid_created_ms=None, session_id=None)

Return a new EdgeRequestContext object made from scratch.

Services at the edge that communicate directly with clients should use this to pass on the information they get to downstream services. They can then use this information to check authentication, run experiments, etc.

To use this, create and attach the context early in your request flow:

auth_cookie = request.cookies["authentication"]
token = request.authentication_service.authenticate_cookie(cookie)
loid = parse_loid(request.cookies["loid"])
session = parse_session(request.cookies["session"])

edge_context = self.edgecontext_factory.new(
    authentication_token=token,
    loid_id=loid.id,
    loid_created_ms=loid.created,
    session_id=session.id,
)
edge_context.attach_context(request)
Parameters:
  • authentication_token – (Optional) A raw authentication token as returned by the authentication service.
  • loid_id (str) – (Optional) ID for the current LoID in fullname format.
  • loid_created_ms (int) – (Optional) Epoch milliseconds when the current LoID cookie was created.
  • session_id (str) – (Optional) ID for the current session cookie.
from_upstream(edge_header)

Create and return an EdgeRequestContext from an upstream header.

This is generally used internally to Baseplate by framework integrations that automatically pick up context from inbound requests.

Parameters:edge_header – Raw payload of Edge-Request header from upstream service.
class baseplate.core.EdgeRequestContext(authn_token_validator, header)

Contextual information about the initial request to an edge service

Construct this using an EdgeRequestContextFactory.

attach_context(context)

Attach this to the provided context object.

Parameters:context – request context to attach this to
event_fields()

Return fields to be added to events.

user

User object for the current context

oauth_client

OAuthClient object for the current context

session

Session object for the current context

service

Service object for the current context

class baseplate.core.User

Wrapper for the user values in AuthenticationToken and the LoId cookie.

id

Authenticated account_id for the current User.

Type:account_id string or None if context authentication is invalid
Raises:NoAuthenticationError if there was no authentication token, it was invalid, or the subject is not an account.
is_logged_in

Does the User have a valid, authenticated id?

roles

Authenticated roles for the current User.

Type:set(string)
Raises:NoAuthenticationError if there was no authentication token or it was invalid
has_role(role)

Does the authenticated user have the specified role?

Parameters:client_types (str) – Case-insensitive sequence role name to check.
Type:bool
Raises:NoAuthenticationError if there was no authentication token defined for the current context
event_fields()

Return fields to be added to events.

class baseplate.core.OAuthClient

Wrapper for the OAuth2 client values in AuthenticationToken.

id

Authenticated id for the current client

Type:string or None if context authentication is invalid
Raises:NoAuthenticationError if there was no authentication token defined for the current context
is_type(*client_types)

Is the authenticated client type one of the given types?

When checking the type of the current OauthClient, you should check that the type “is” one of the allowed types rather than checking that it “is not” a disallowed type.

For example:

if oauth_client.is_type("third_party"):
    ...

not:

if not oauth_client.is_type("first_party"):
    ...
Parameters:client_types (str) – Case-insensitive sequence of client type names that you want to check.
Type:bool
Raises:NoAuthenticationError if there was no authentication token defined for the current context
event_fields()

Return fields to be added to events.

class baseplate.core.Session(id)
class baseplate.core.AuthenticationToken

Information about the authenticated user.

EdgeRequestContext provides high-level helpers for extracting data from authentication tokens. Use those instead of direct access through this class.

subject

The raw subject that is authenticated.

exception baseplate.core.NoAuthenticationError

Raised when trying to use an invalid or missing authentication token.