pushcollector

A library for collecting information from push tasks.

Contents:

• API Reference
  • Collector
• Backends
  • local
  • dummy
  • Implementing a backend
• Push Item Schema

Quick Start

Install pushcollector from PyPI:

pip install pushcollector

In your python code, obtain a Collector instance and use it to record information about a task as it runs:

import sys
from concurrent import futures
from pushcollector import Collector

collector = Collector.get()

fs = []
items = []
for filename in sys.argv[1:]:
    try:
        upload_file(filename)
        items.append({"filename": filename, "state": "PUSHED"})
    except Exception as exc:
        log.exception("Error handling %s", filename)
        items.append({"filename": filename, "state": "UPLOADFAILED"})
        fs.append(collector.append_file('error.log', str(exc)))

fs.append(collector.update_push_items(items))

# Wait for the collector to finish all work
futures.wait(fs)

The Collector class will record data using the configured backend. pushcollector provides a couple of simple backends, but it will often be necessary to implement a custom backend appropriate for your task execution environment. See Backends for more information about available backends and implementing your own backend.