Coordinating Goroutine Lifecycles with WaitGroups and ErrGroups

Go is famous for making concurrency feel effortless through the use of the go keyword. However, spawning a goroutine is only the beginning of building a production-ready system. The real challenge lies in coordination, ensuring that your main process does not exit before your background workers have finished their tasks. In a production environment, an uncoordinated exit can lead to corrupted data, half-written files, or lost network requests.

While channels are a powerful way to pass data between goroutines, using them solely for synchronization can often feel verbose and clumsy. If your only goal is to wait for a set of workers to complete, the sync package provides a more streamlined primitive called the WaitGroup. This tool acts as a synchronized counter that allows the main thread to block until every individual worker has signaled that its work is done.

The mental model for a WaitGroup is similar to a ticket counter at a stadium entrance. You increment the counter for every person who enters the building and decrement it as they exit. The stadium cannot be closed for the night until the counter returns to exactly zero. This simple counting mechanism provides a robust way to manage the lifecycle of parallel processes without the overhead of channel buffers or complex signaling logic.

1package main
2
3import (
4	"fmt"
5	"sync"
6	"time"
7)
8
9func processBatch(id int, wg *sync.WaitGroup) {
10	// Use defer to ensure Done is called even if a panic occurs
11	defer wg.Done()
12
13	fmt.Printf("Worker %d: Starting image transformation...\n", id)
14	time.Sleep(time.Second) // Simulate heavy CPU work
15	fmt.Printf("Worker %d: Image processed successfully\n", id)
16}
17
18func main() {
19	var wg sync.WaitGroup
20	images := []int{101, 102, 103, 104, 105}
21
22	for _, id := range images {
23		// Increment before starting the goroutine to prevent a race
24		wg.Add(1)
25		go processBatch(id, &wg)
26	}
27
28	// Wait blocks until the internal counter reaches zero
29	wg.Wait()
30	fmt.Println("Main: All images processed. Exiting.")
31}

The implementation details of a WaitGroup are deceptively simple but require discipline to use correctly. One of the most common mistakes is calling the Add method inside the spawned goroutine rather than before it. If the scheduler delays the execution of the goroutine, the main routine might reach the Wait call while the counter is still zero, causing the program to exit prematurely. Always call Add in the calling routine to guarantee the counter reflects the intended workload immediately.

While a standard WaitGroup is excellent for tracking completion, it has a significant limitation: it does not handle errors. In a real-world application, such as an API aggregator or a cloud resource provisioner, it is rare for all parallel tasks to succeed every time. If one worker fails, the WaitGroup simply continues waiting for the rest, leaving the developer to find a manual way to collect and report those failures.

The errgroup package, part of the Go extended library, was designed to solve this exact problem. It provides a more sophisticated version of the WaitGroup that tracks the success or failure of each goroutine. It simplifies error propagation by capturing the first error that occurs and returning it to the main caller, allowing for a more graceful and responsive error handling strategy.

Beyond just collecting errors, an ErrGroup can also be used to implement short-circuiting. By combining it with the context package, an ErrGroup can automatically cancel all other active tasks the moment one of them fails. This prevents your system from wasting valuable CPU cycles and network bandwidth on a batch job that has already failed a critical step.

1package main
2
3import (
4	"context"
5	"errors"
6	"fmt"
7	"golang.org/x/sync/errgroup"
8	"net/http"
9)
10
11func fetchStatus(ctx context.Context, url string) error {
12	req, _ := http.NewRequestWithContext(ctx, "GET", url, nil)
13	resp, err := http.DefaultClient.Do(req)
14	if err != nil {
15		return err // Errors are automatically captured by the group
16	}
17	defer resp.Body.Close()
18
19	if resp.StatusCode != http.StatusOK {
20		return errors.New("failed to reach service: " + url)
21	}
22	return nil
23}
24
25func main() {
26	g, ctx := errgroup.WithContext(context.Background())
27	urls := []string{"https://api.one.com", "https://api.two.com"}
28
29	for _, url := range urls {
30		url := url // Capture loop variable
31		g.Go(func() error {
32			return fetchStatus(ctx, url)
33		})
34	}
35
36	if err := g.Wait(); err != nil {
37		fmt.Printf("Aggregation failed: %v\n", err)
38		return
39	}
40	fmt.Println("All services are healthy")
41}

In some scenarios, goroutines are not just performing isolated tasks but are actively modifying a shared piece of state. For instance, if you are building a high-frequency trading system or a real-time analytics dashboard, multiple threads might attempt to update a central cache or a counter at the same time. Accessing the same memory location from multiple goroutines simultaneously creates a race condition.

A race condition is particularly dangerous because it does not always cause a crash during development. Instead, it leads to silent data corruption or unpredictable behavior that only appears under high production load. To prevent this, the sync package offers a Mutex, which stands for mutual exclusion, to ensure that only one goroutine can access a critical section of code at a time.

The golden rule of Go concurrency is to share memory by communicating, but when that is not feasible, use a Mutex to ensure that your shared memory is never accessed by two threads at the same time. Performance without correctness is simply a faster way to get the wrong answer.

A Mutex acts like a physical lock on a room. Before a goroutine can read from or write to a protected variable, it must first acquire the lock. If another goroutine already holds the lock, the second goroutine will block and wait its turn. Once the first goroutine is finished and releases the lock, the waiting goroutine can proceed, ensuring total data integrity.

1package main
2
3import (
4	"fmt"
5	"sync"
6)
7
8type SecureCache struct {
9	mu    sync.Mutex
10	items map[string]string
11}
12
13func (c *SecureCache) Update(key, value string) {
14	c.mu.Lock()         // Acquire exclusive access
15	defer c.mu.Unlock() // Ensure release even if logic changes
16	c.items[key] = value
17}
18
19func (c *SecureCache) Fetch(key string) string {
20	c.mu.Lock()
21	defer c.mu.Unlock()
22	return c.items[key]
23}
24
25func main() {
26	cache := &SecureCache{items: make(map[string]string)}
27	var wg sync.WaitGroup
28
29	for i := 0; i < 10; i++ {
30		wg.Add(1)
31		go func(v int) {
32			defer wg.Done()
33			key := fmt.Sprintf("key-%d", v)
34			cache.Update(key, "processed")
35		}(i)
36	}
37
38	wg.Wait()
39	fmt.Println("Cache updates complete and race-free")
40}