Deadlock on All GoRoutines When Using Channels and WaitGroup

February 7, 2022

I am new to Go and am currently attempting to run a function that creates a file and returns it’s filename and have this run concurrently.

I’ve decided to try and accomplish this with goroutines and a WaitGroup. When I use this approach, I end up with a list size that is a couple hundred files less than the input size. E.g. for 5,000 files I get around 4,700~ files created.

I believe this is due to some race conditions:

wg := sync.WaitGroup{}

filenames := make([]string, 0)

for i := 0; i < totalFiles; i++ {
    wg.Add(1)
    go func() {
        defer wg.Done()
        filenames = append(filenames, createFile())
    }()
}

wg.Wait()

return filenames, nil

Don’t communicate by sharing memory; share memory by communicating.

I tried using channels to "share memory by communicating". Whenever I do this, there appears to be a deadlock that I can’t seem to wrap my head around why. Would anyone be able to point me in the right direction for using channels and waitgroups together properly in order to save all of the created files to a shared data structure?

This is the code that produces the deadlock for me (fatal error: all goroutines are asleep – deadlock!):

wg := sync.WaitGroup{}

filenames := make([]string, 0)
ch := make(chan string)

for i := 0; i < totalFiles; i++ {
    wg.Add(1)
    go func() {
        defer wg.Done()
        ch <- createFile()
    }()
}

wg.Wait()

for i := range ch {
    filenames = append(filenames, i)
}

return filenames, nil

Thanks!

>Solution :

The first one has a race. You have to protect access to filenames:

mu:=sync.Mutex{}
for i := 0; i < totalFiles; i++ {
    wg.Add(1)
    go func() {
        defer wg.Done()
        mu.Lock()
        defer mu.Unlock()
        filenames = append(filenames, createFile())
    }()
}

For the second case, you are waiting for the goroutines to finish, but goroutines can only finish once you read from the channel, so deadlock. You can fix it by reading from the channel in a separate goroutine.

go func() {
  for i := range ch {
      filenames = append(filenames, i)
  }
}()

wg.Wait()
close(ch) // Required, so the goroutine can terminate

return filenames, nil