pikatasks

pikatasks is a minimalistic library that allows you to run remote tasks easily. There's also a Django integration.

Requirements

• pip install pika
• RabbitMQ as message broker

How-to

Import:

import pikatasks

Configure:

pikatasks.settings.BROKER_HOST = "localhost"
pikatasks.settings.BROKER_PORT = 5672 # change to 5671 if using SSL
pikatasks.settings.SSL_ENABLED = False  # change to True in production :)
pikatasks.settings.VIRTUAL_HOST = "/"  # rabbitmq default vhost
pikatasks.settings.USERNAME = "admin"  # too lazy to change!
pikatasks.settings.PASSWORD = "qwertz"  # easy to remember!

Or in Django settings:

PIKATASKS_BROKER_HOST = "localhost"
PIKATASKS_BROKER_PORT = 5672
...

Implement a task (server):

@pikatasks.task
def hello(something):
    msg = "Hello, " + something + "!"
    print(msg)
    return msg

• The task name (and the queue name) will be the same as the function name. If you want to specify a custom task (and queue) name, use @pikatasks.task("my_task_name").
• Note: you will need a queue with exactly the same name as the task. See section: Queues and Permissions.

Start a server:

pikatasks.worker.start()

Run a task (client):

To simply run a task:

pikatasks.run("hello", something="World")

Run a task and get its result:

result = pikatasks.rpc("hello", something="World")
print(result)
#  >>> "Hello, World!"

Catch exceptions:

if a task raises an exception on the server, pikatask.rpc() call on the client will also raise an exception. Full exception message is not sent for security/isolation reasons.

try:
  pikatasks.run("hello", something=42)
except pikatasks.RPCError as e:
  print(e)
  # >>> Task hello raised TypeError (see worker log for details).
  # Note: TypeError was raised when the server was running: "Hello, " + 42 + "!"

Queues and Permissions

Queues and exchanges:

With AMQ, messages first arrive to exchanges, then broker distributes them to to queues using routing keys. If you are not sure what it is all about, read this tutorial first and further RabbitMQ documentation if needed.

Queues and Tasks:

• Pikatasks requires a separate queue for each task.
• queue name == task name
• You need to create these queues by yourself (e.g. using the RabbitMQ Management Plugin web interface).

Developent setup:

You are done after creating queues for each of your tasks. Don't need anything else for the development. Note: exchange amq.default will be used.

Client:

• Create a new exchange for your client. Let's call it client.out, and its type should be direct. This exchange will be used for sending tasks.
• Decide which tasks should the client use. Let's say these are task1 and task2 (you should have the corresponding queues already).
• For each of the tasks, create a new binding for the exchange client.out, with routing key == queue name == task name
  • e.g. exchange = client.out, routing key = task1, queue = task1
• RabbitMQ user permissions:
  • Configure: empty string (no config permissions)
  • Write: ^client.out$ (replace with the name of your exchange)
  • Read: empty string (no read permissions, RPC results/replies will still work)
• Pikatasks settings:
  • pikatasks.settings.CLIENT_EXCHANGE_NAME = "client.out" (replace client.out with you know what`)

Worker:

• RabbitMQ user permissions:
  • Configure: empty string (no config permissions)
  • Write: .* (everything) or ^amq.default$ (amq.default is required to send "direct reply-to")
  • Read: ^(task1|task2)$, replace taskN with whatever your task names are