Add support for RabbitMQ to master.

[GoldenBadger]

RabbitMQ is a message broker which we will use to decouple the master and workers. Instead of the master keeping track of each worker individually, it will send tasks to the broker which will each be assigned to a single worker.

Marked P0 as this will be the sole method by which the master will communicate with the workers.

Child of #345

[GoldenBadger]

Proposed API

Code

trait Broker {
    fn connect(addr: std::net::SocketAddr) -> Result<BrokerConnection, Error>;
}

struct BrokerConnection {
    handle: futures::sync::mpsc::Sender<Task>,
    error_future: Box<Future<Item = (), Error = Error>>,
}

Explanation

Broker::connect takes an address to a message broker server and attempts to connect to it. If the connection is successful it returns a BrokerConnection.

The BrokerConnection contains a future for error handling (see #363) and a handle to a MPSC channel. All Tasks submitted to the channel are sent to the message broker. The handle is cloneable and can be sent across threads.

[GoldenBadger]

@VoyTechnology Review pls

[VoyTechnology]

It might be a more general question, but how would we handle cancelling of jobs? Is there a command for RabbitMQ to tell it to stop restarting tasks?

[GoldenBadger]

I'm not sure, but if a job is cancelled then just not sending any new tasks should be enough. Overall it shouldn't have a huge impact leaving a few Jobless Tasks running to completion. Then we can just ignore their results. It's something to think about.

[VoyTechnology]

Never followed up. SGTM