Using AWS

RelengAPI provides a light wrapper around Boto for access to AWS resources. The wrapper is accessible at current_app.aws, which is an instance of relengapi.lib.aws.AWS:

SQS

Amazon SQS provides simple queues with read and write operations. From within a RelengAPI application, writing to a queue is trivial (app.aws.sqs_write). However, reading from a queue requires a dedicated thread for that operation, and that thread does not run in the same process as the HTTP service. Instead, all SQS readers run in a process started with relengapi sqs-listen. See AWS SQS for more information.

Registering a listener in a blueprint can be a little tricky, since the decorator is connected to the application. The application is available through @bp.record:

@bp.record
def init_blueprint(state):
    app = state.app
    @app.aws.sqs_listen('us-west-2', 'my-queue')
    def my_queue_msg(msg):
        .. # process the message

Note that this pattern also makes it easy to get the region and queue name from the blueprint configuration. Hard-coding this information is not a good idea!

AWS Class

class relengapi.lib.aws.AWS

Generic methods:

connect_to(service_name, region_name)
Parameters:
  • service_name (string) -- name of the service to connect to (e.g., sqs)

  • region_name (string) -- name of the region in which to connect (e.g., us-west-2)

Returns:

Boto connection instance

This low-level method wrapps the various boto.connect_foo methods, handling authentication, regions, and caching of connections.

SQS-related methods:

get_sqs_queue(region_name, queue_name)
Parameters:
  • region_name (string) -- name of the region in which to connect (e.g., us-west-2)

  • queue_name (string) -- name of the queue

Returns:

Boto Queue instance

Fetches the configuration for the named queue, then gets the corresponding boto Queue instance. Subsequent operations on the queue should use the Boto interface directly. In most cases, you'll want sqs_write() instead

sqs_write(region_name, queue_name, body)
Parameters:
  • region_name (string) -- name of the region in which to connect (e.g., us-west-2)

  • queue_name (string) -- name of the queue

  • body -- JSON-able data to be placed in the message body

Writes body to the given queue. This uses WSME to JSON-ify the body, after which Boto base64-encodes the JSON content as AWS recommends. If possible, blueprints should use the same WSME types as the HTTP responses for SQS messages.

To send messages with some other format (for example, without base64 encoding, or as simple strings), use get_sqs_queue() to get a Queue instance, then construct and send the Message directly.

sqs_listen(region_name, queue_name, read_args)
Parameters:
  • region_name (string) -- name of the region in which to connect (e.g., us-west-2)

  • queue_name (string) -- name of the queue

  • read_args (dictionary) -- arguments to the boto Queue.read method

Decorate the following function to receive messages from the named queue.

The function will be called in the context of the RelengAPI application in the relengapi sqs-listen process. The read_args are passed as keyword arguments to Queue.read, although wait_time_seconds is not available (it is already set).