Our platform consists of hundreds of Python agents running on EC2 instances in an ASG that we wish to monitor with Prometheus. The agents will generally be long-running (days, weeks, months), but can be killed or migrated to another node without warning.

Because of this, it is not possible to use a fixed port number for exposing the Prometheus endpoint. However, we can pass 0 as the port number and let the OS find an ephemeral port to listen on. Unfortunately there doesn't seem to be a way to surface the resulting port number back to the agent so that we can then populate Consul and let Prometheus know where to scrape data from.

This PR addresses this shortcoming.

from prometheus_client import start_http_server, Summary
import random
import time

# Create a metric to track time spent and requests made.
REQUEST_TIME = Summary('request_processing_seconds', 'Time spent processing request')

# Decorate function with metric.
@REQUEST_TIME.time()
def process_request(t):
    """A dummy function that takes some time."""
    time.sleep(t)

if __name__ == '__main__':
    # Start up the server to expose the metrics.
    server_address = start_http_server(0)
    print(server_address) # or send to Consul
    # Generate some requests.
    while True:
        process_request(random.random())