Skip to content
Snippets Groups Projects
README.md 3.73 KiB
Newer Older
  • Learn to ignore specific revisions
  • Art's avatar
    Art committed
    # pikatasks
    
    
    **pikatasks** is a minimalistic library that allows you to run remote tasks easily. There's also a Django integration.
    
    Art's avatar
    Art committed
    
    
    ## Requirements
      - `pip install pika`
      - RabbitMQ as message broker
    
    
    ## How-to
    ##### Import:
    ```python
    import pikatasks
    ```
    
    ##### Configure:
    ```python
    
    pikatasks.settings.BROKER_HOST = "localhost"
    
    Art's avatar
    Art committed
    pikatasks.settings.BROKER_PORT = 5672 # change to 5671 if using SSL
    pikatasks.settings.SSL_ENABLED = False  # change to True in production :)
    pikatasks.settings.VIRTUAL_HOST = "/"  # rabbitmq default vhost
    pikatasks.settings.USERNAME = "admin"  # too lazy to change!
    pikatasks.settings.PASSWORD = "qwertz"  # easy to remember!
    
    Art's avatar
    Art committed
    ```
    
    Or in Django settings:
    ```python
    
    PIKATASKS_BROKER_HOST = "localhost"
    
    Art's avatar
    Art committed
    PIKATASKS_BROKER_PORT = 5672
    ...
    
    Art's avatar
    Art committed
    ##### Implement a task (server):
    ```python
    
    @pikatasks.task
    
    Art's avatar
    Art committed
    def hello(something):
        msg = "Hello, " + something + "!"
        print(msg)
        return msg
    ```
    
      - The task name (and the queue name) will be the same as the function name. If you want to specify a custom task (and queue) name, use `@pikatasks.task("my_task_name")`.
      - Note: you will need a queue with exactly the same name as the task. See section: Queues and Permissions.
    
    Art's avatar
    Art committed
    
    ##### Start a server:
    ```python
    pikatasks.worker.start(tasks=[hello])
    ```
    
    ##### Run a task (client):
    To simply run a task:
    ```python
    pikatasks.run("hello", something="World")
    ```
    
    Art's avatar
    Art committed
    Run a task and get its result:
    
    Art's avatar
    Art committed
    ```python
    result = pikatasks.rpc("hello", something="World")
    print(result)
    #  >>> "Hello, World!"
    ```
    
    ##### Catch exceptions:
    
    if a task raises an exception on the server, `pikatask.rpc()` call on the client will also raise an exception. Full exception message is not sent for security/isolation reasons.
    
    ```python
    try:
      pikatasks.run("hello", something=42)
    except pikatasks.RPCError as e:
      print(e)
      # >>> Task hello raised TypeError (see worker log for details).
      # Note: TypeError was raised when the server was running: "Hello, " + 42 + "!"
    ```
    
    ## Queues and Permissions
    ##### Queues and exchanges:
    With AMQ, messages first arrive to `exchanges`, then broker distributes them to to `queues` using `routing keys`. If you are not sure what it is all about, read [this tutorial](https://www.rabbitmq.com/tutorials/tutorial-four-python.html) first and further RabbitMQ documentation if needed.
    
    ##### Queues and Tasks:
      * ***pikatasks*** requires a separate queue for each task.
      * `queue name == task name`
      * You need to create these queues by yourself.
    
    ##### Developent setup:
    You are done after creating queues for each of your tasks. Don't need anything else for the development. Note: exchange `amq.default` will be used.
    
    ##### Client:
    
      * Create a new exchange for your client. Let's call it `client.out`, and its type should be `direct`. This exchange will be used for sending tasks.
    
    Art's avatar
    Art committed
      * Decide which tasks should the client use. Let's say these are `task1` and `task2` (you should have the corresponding queues already).
    
      * For each of the tasks, create a new binding for the exchange `client.out`, with `routing key == queue name == task name`
        * e.g. `exchange = client.out`, `routing key = task1`, `queue = task1`
    
    Art's avatar
    Art committed
      * RabbitMQ user permissions:
    
        * Configure: empty string (no config permissions)
    
        * Write: `^client.out$` (replace with the name of your exchange)
    
        * Read: empty string (no read permissions, RPC results/replies will still work)
    
    Art's avatar
    Art committed
      * Pikatasks settings:
        * ```pikatasks.settings.CLIENT_EXCHANGE_NAME = "client.out"``` (replace `client.out` with you know what`)
    
    Art's avatar
    Art committed
    
    ##### Worker:
    
    Art's avatar
    Art committed
      * RabbitMQ user permissions:
    
        * Configure: empty string (no config permissions)
        * Write: `.*` (everything) or `^amq.default$` (`amq.default` is required to send "direct reply-to")
    
    Art's avatar
    Art committed
        * Read: `^(task1|task2)$`, replace `taskN` with whatever your task names are