Go by Example: Worker Pools
|
In this example we’ll look at how to implement a worker pool using goroutines and channels. |
|
package main
|
|
import "fmt"
import "time"
|
|
|
Here’s the worker, of which we’ll run several
concurrent instances. These workers will receive
work on the |
func worker(id int, jobs <-chan int, results chan<- int) {
for j := range jobs {
fmt.Println("worker", id, "started job", j)
time.Sleep(time.Second)
fmt.Println("worker", id, "finished job", j)
results <- j * 2
}
}
|
func main() {
|
|
|
In order to use our pool of workers we need to send them work and collect their results. We make 2 channels for this. |
jobs := make(chan int, 100)
results := make(chan int, 100)
|
|
This starts up 3 workers, initially blocked because there are no jobs yet. |
for w := 1; w <= 3; w++ {
go worker(w, jobs, results)
}
|
|
Here we send 5 |
for j := 1; j <= 5; j++ {
jobs <- j
}
close(jobs)
|
|
Finally we collect all the results of the work. |
for a := 1; a <= 5; a++ {
<-results
}
}
|
|
Our running program shows the 5 jobs being executed by various workers. The program only takes about 2 seconds despite doing about 5 seconds of total work because there are 3 workers operating concurrently. |
$ time go run worker-pools.go
worker 1 started job 1
worker 2 started job 2
worker 3 started job 3
worker 1 finished job 1
worker 1 started job 4
worker 2 finished job 2
worker 2 started job 5
worker 3 finished job 3
worker 1 finished job 4
worker 2 finished job 5
|
real 0m2.358s
|
Next example: Rate Limiting.