Task Distribution Tasks with ZeroMQ
Learn how to run parallel tasks using a messaging pipeline and ZeroMQ.
We’re now starting to understand how ZeroMQ is different from traditional web services and why it’s a perfect tool for building a distributed messaging system.
Building a distributed hashsum cracker with ZeroMQ
Now it’s time to build a sample application to see the properties of the PUSH
/PULL
sockets we just described in action.
A simple and fascinating application to work with would be a hashsum cracker: A system that uses a brute-force approach to try to match a given hashsum (such as MD5 or SHA1) to the hashsum of every possible variation of characters of a given alphabet, therefore discovering the original string that the given hashsum was created from.
This is an embarrassingly parallel workload, which is perfect for building an example demonstrating the power of parallel pipelines.
Note: Never use plain hashsums to encrypt passwords as they’re very easy to crack. Instead, use ia purpose-built algorithm such as bcrypt, scrypt, PBKDF2, or Argon2.
For our application, we want to implement a typical parallel pipeline where we have the following:
A node to create and distribute tasks across multiple workers
Multiple worker nodes (where the actual computation happens)
A node to collect all the results
The system we just described can be implemented in ZeroMQ using the following architecture:
Get hands-on with 1200+ tech skills courses.