Reduce Scheduling Contention
Explore techniques to enhance the performance of Go command-line tools by reducing scheduling contention. Understand how to implement worker queues using goroutines matching the number of CPU cores, manage job distribution via channels, and improve efficiency in CPU-bound tasks. This lesson helps you optimize your Go programs for faster execution and better resource management.
We'll cover the following...
Overview
Goroutines are relatively cheap, and in some cases it makes sense to create many of them. For example, if the goroutines wait a long time on IO or network response, then the scheduler can execute other goroutines while these are waiting, increasing the efficiency of the program. In our case, the goroutines are mostly CPU-bound, so creating many of them doesn’t improve the efficiency. Instead, creating too many causes scheduling contention. Let’s address this issue by modifying our program to use worker queues. Instead of creating one goroutine per file, we’ll create one goroutine per available CPU. These will be our workers. Another goroutine sends the jobs to be executed by the workers. When no more jobs exist, the workers are done and the program finishes.
We ...