Coding Challenge: Queue-Based Distribution

Problem

Suppose you have a bunch of documents, and you want to count the frequency of all the words occurring in those documents. This is a heavy task so you want to use a queue-based system celery with the broker and backend of redis.

For simplicity, you can assume that we only have three documents whose text is stored in the list named texts beforehand. Now, your task is to define a function getWordCount(text) that takes a text as an argument and returns the dictionary of words and their frequencies. You have to register that function with celery and deploy a worker to execute the tasks.

For your convenience, we have provided the skeleton code and also started the redis backend ourselves.

To run the code (celery and worker) in the playground below, run python celery-test.py in the terminal. Then, open another terminal and enter command cd examples/queue-based-distribution/ to move to the directory. Then, enter the command: celery -A celery-test worker.