使用Golang的同步和异步工作者实例

169 阅读4分钟

在这个例子中,我们将创建一个工作者包,它将用于同步和异步作业的处理情况。

尽管这个包做到了它所承诺的,但它还需要三个额外的功能:

  • 终止长期运行的作业的能力

  • 关闭通道的能力

  • 能够杀死所有的工作者

这个例子使用了 "无缓冲 "的工作通道。然而,如果你想限制作业的时间,你可以更新NewWorker() ,使用 "缓冲 "通道,如下所示:

func NewWorker(workerTotal, jobTotal int) Worker {
	return Worker{
		total:      workerTotal,
		jobChan:    make(chan Job, jobTotal),
		resultChan: make(chan interface{}, jobTotal),
	}
}
package worker

import (
	"fmt"
	"math/rand"
	"time"
)

// -----------------------------------------------------------------------------

type Job struct {
	// id represents the job identifier.
	id interface{}
}

// NewJob returns a `Job` instance.
func NewJob(id interface{}) Job {
	return Job{
		id: id,
	}
}

// -----------------------------------------------------------------------------

type Worker struct {
	// total represents the amount of workers to be run at startup.
	total int
	// jobChan represents a two-way "unbuffered" channel that has unlimited
	// capacity for the jobs.
	jobChan chan Job
	// resultChan represents a two-way "unbuffered" channel that has unlimited
	// capacity for the job results.
	resultChan chan interface{}
}

// NewWorker returns a `Worker` instance.
func NewWorker(workerTotal int) Worker {
	return Worker{
		total:      workerTotal,
		jobChan:    make(chan Job),
		resultChan: make(chan interface{}),
	}
}

// Start brings up certain amount of worker(s) so that they can pick up and work
// on the job(s).
func (w Worker) Start() {
	for i := 1; i <= w.total; i++ {
		go w.run(i)
	}
}

// Add adds a job to a channel so that it could be picked up and worked on by
// the running worker(s).
func (w Worker) Add(job Job) {
	w.jobChan <- job
}

// Result returns a channel so that it could be ranged over in order to fetch
// job results from the running worker(s).
func (w Worker) Result() <-chan interface{} {
	return w.resultChan
}

// run runs a worker and works on the job(s).
func (w Worker) run(id int) {
	fmt.Println(id, "running...")

	for {
		select {
		case job := <- w.jobChan:
			fmt.Printf("%d picked up job %v @ %s\n", id, job.id, time.Now().UTC())

			// Pretend like doing something.
			rand.Seed(time.Now().UnixNano())
			time.Sleep(time.Duration(rand.Intn(len([]int{0, 1, 2, 3, 4}))) * time.Second)
			// Done.

			fmt.Printf("%d completed job %v @ %s\n", id, job.id, time.Now().UTC())
			w.resultChan <- job.id
		default:
			time.Sleep(1 * time.Second)
			fmt.Println(id, "waiting...")
		}
	}
}

使用方法

异步的

这是一个异步(非阻塞)的例子,所以作业是随机处理的,没有顺序。程序从不退出。作业结果是独立打印的。拥有多个工作者是很重要的。

package main

import (
	"fmt"

	"internal/worker"
)

func main() {
	// Create new worker(s) and start.
	w := worker.NewWorker(3)
	w.Start()

	go func() {
		// Add jobs.
		for i := 1; i <= 5; i++ {
			w.Add(worker.NewJob(i))
		}
	}()

	// Print results.
	for v := range w.Result() {
		fmt.Println("Result:", v)
	}
}
1 running...
2 running...
3 running...
1 picked up job 1 @ 2020-04-11 18:47:12.496663 +0000 UTC
1 completed job 1 @ 2020-04-11 18:47:12.496757 +0000 UTC
Result: 1
1 picked up job 2 @ 2020-04-11 18:47:12.49679 +0000 UTC
2 waiting...
3 waiting...
2 picked up job 3 @ 2020-04-11 18:47:13.496786 +0000 UTC
2 completed job 3 @ 2020-04-11 18:47:13.496839 +0000 UTC
3 picked up job 4 @ 2020-04-11 18:47:13.496813 +0000 UTC
Result: 3
1 completed job 2 @ 2020-04-11 18:47:14.497218 +0000 UTC
1 picked up job 5 @ 2020-04-11 18:47:14.497286 +0000 UTC
Result: 2
2 waiting...
2 waiting...
1 completed job 5 @ 2020-04-11 18:47:16.497519 +0000 UTC
Result: 5
2 waiting...
1 waiting...
3 completed job 4 @ 2020-04-11 18:47:17.501962 +0000 UTC
Result: 4
2 waiting...
1 waiting...
3 waiting...
2 waiting...
1 waiting...
3 waiting...
2 waiting...
1 waiting...

同步的

这是一个同步(阻塞)的例子,所以作业是按顺序一个接一个地处理的。程序在作业完成后退出。作业结果按顺序一个接一个地打印。拥有多个工作者是没有意义的,因为一次只能处理一个作业。

package main

import (
	"fmt"

	"internal/worker"
)

func main() {
	// Create new worker(s) and start.
	w := worker.NewWorker(3)
	w.Start()

	// Add jobs.
	w.Add(worker.NewJob(1))
	// Print results.
	v := <- w.Result()
	fmt.Println("Result:", v)

	// Add jobs.
	w.Add(worker.NewJob(2))
	// Print results.
	v = <- w.Result()
	fmt.Println("Result:", v)

	// Add jobs.
	w.Add(worker.NewJob(3))
	// Print results.
	v = <- w.Result()
	fmt.Println("Result:", v)

	// Add jobs.
	w.Add(worker.NewJob(4))
	// Print results.
	v = <- w.Result()
	fmt.Println("Result:", v)

	// Add jobs.
	w.Add(worker.NewJob(5))
	// Print results.
	v = <- w.Result()
	fmt.Println("Result:", v)
}
1 running...
3 running...
2 running...
1 picked up job 1 @ 2020-04-11 18:52:14.135716 +0000 UTC
3 waiting...
2 waiting...
3 waiting...
2 waiting...
2 waiting...
3 waiting...
1 completed job 1 @ 2020-04-11 18:52:18.137946 +0000 UTC
Result: 1
3 waiting...
2 waiting...
3 picked up job 2 @ 2020-04-11 18:52:18.149216 +0000 UTC
1 waiting...
2 waiting...
1 waiting...
3 completed job 2 @ 2020-04-11 18:52:21.150342 +0000 UTC
Result: 2
2 waiting...
2 picked up job 3 @ 2020-04-11 18:52:21.153484 +0000 UTC
2 completed job 3 @ 2020-04-11 18:52:21.153534 +0000 UTC
Result: 3
1 waiting...
1 picked up job 4 @ 2020-04-11 18:52:22.144573 +0000 UTC
3 waiting...
2 waiting...
2 waiting...
3 waiting...
2 waiting...
1 completed job 4 @ 2020-04-11 18:52:26.146923 +0000 UTC
Result: 4
3 waiting...
2 waiting...
3 picked up job 5 @ 2020-04-11 18:52:26.164251 +0000 UTC
1 waiting...
2 waiting...
3 completed job 5 @ 2020-04-11 18:52:27.167167 +0000 UTC
Result: 5