Defer

Defer is another of Go’s unique features. In this post I look at how and why to use it, how it interacts with return and panic and a few potential pitfalls.

Not freeing resources is a bug that can easily go undetected since the consequences are not always, or not at all, obvious. This problem has been tackled in other languages (as I discuss in the Background section below) but Go’s solution is by far the best I have seen. (It’s also another great example of what I call DIRE.

Background

General

C

C programs are notorious for having resource leaks. After decades of using C I came to be pretty good at remembering to add code to clean up after myself, including writing assertions and tests to check these things. However, I am sure every C programmer (including me) have at some point written code that would leak something (memory, window resource, …) for some error(s) - especially error conditions without unit tests.

Disregarding that, even the most diligent programmer cannot protect their code from future changes. For example, an early return from a function might be added which forgets to close a file handle.

You have probably seen this sort of thing when using an editor: you can’t understand why the OS won’t let you delete a file since you had already closed the file in your favourite editor. However, once you shut down the editor completely, you can delete the file. The editor (probably written in C :) had omitted to close the file handle.

C programmers adopt various strategies to mitigate this problem. One set of “coding standards” I was forced to work with insisted that all functions have one exit point (at the end). This made it easier to check that resources were freed (and had other advantages like setting breakpoints in the debugger) but it does cause less readable and more complicated code.

It’s difficult, in C, to ensure that everything is cleaned up in all code paths. It’s impossible to ensure that future modifications do the same.

C++

In C++ things are a bit easier than C, since the language has constructors and destructors. Wrapping the resource in an object you can allocate in the constructor and free it in the destructor. This was formalised by Bjarne Stroustrup with the acronym RAII (resource acquisition is initialization).

Like Go’s defer this has the advantage that an early return or panic (throw in C++) cannot bypass the cleanup code. It also has the important (DIRE) advantage that the allocate/de-allocate code can be kept together - assuming that the destructor is placed just after the constructor(s) as is conventionally done.

Java

Java and C# have the closest thing that I have seen to Go’s defer. A finally block of code after a try statement can be used to ensure that code cannot be bypassed using an early return or throw. (Note that this operates at the try statement level, not the function level as in Go.)

Since you must use try to use finally many Java developers think this is only for use with exceptions, but it can actually be used even in the absence of exceptions.

The only problem with finally is that the “deallocate” code becomes separated from the “allocate” code by the body of the try statement, with DIRE consequences.

---

Go

Basics of defer

As already mentioned Go’s defer mechanism is great for ensuring that resources are freed. But it also has some interesting interactions with other parts of the Go language, like panic and recover. I’ll look at these, and other nuances, later, assuming you already know these basics:

• defer applies at the function not statement level
• funcs are (obviously) run in reverse order they were deferred
• deferred functions are called even if the function panics
• deferred functions are invoked after the return statement has been executed
• panics may only be recover()ed inside a deferred func

RAII vs Go’s Recover

Not being an OO language, Go cannot use RAII because there are no destructors. (Go does have finalizers but these are of no use for RAII since they are only called when the object’s memory is freed, which can be much later.)

Luckily, Go has full support for functional programming, which allowed the simple solution of the defer statement. I find defer somehow simpler and more flexible,

**Exceptions vs Go’s Panic

In Go panic is the analogue of throw in other languages. However, catching an “exception” (using recover) is different - you can only recover from panic in a deferred function. At first, this seemed to me like an unnecessary complication but seems to avoid a lot of problems I had in C++ such as exceptions being thrown from inside a destructor which is being called in the context of an exception.

---

A lot has been written on defer in other fine blogs such as Demystifying ‘defer’ so I won’t go over the basics. (If you don’t know what I mean by basics then see the above Background -> Go section.)

Always Use Defer

My first rule is always use defer. It is often tempting not do so. For example:

func (d *Destination) Close(name string) {
    d.mu.Lock()
    if !d.closed {
        d.connection.Close()
        d.closed = true
    }
    d.mu.Unlock() // NOT recommended (for 2 reasons)
}

I have seen a lot of code written like this. So what is wrong with it? What if someone decides to refactor the code like this:

func (d *Destination) Close(name string) {
    d.mu.Lock()
    if d.closed {
        return
    }
    d.connection.Close()
    d.closed = true
    d.mu.Unlock()
}

Now the mutex will not be unlocked if the connection has already been closed. This is a simple example, but these sorts of things happen in realistic code.

But there is a further problem with the original example. It will also not unlock the mutex if d.connection.Close() panics. Then if another attempt is made to close (or do something else with d) the goroutine that calls d.mu.Lock() will block and possibly deadlock everything.

So that’s two good reasons to always use defer.

func (d *Destination) Close(name string) {
    d.mu.Lock()
    defer d.mu.Unlock()
    if !d.closed {
        d.connection.Close()
        d.closed = true
    }
}

Function Split

Here’s another scenario… Sometimes you may be deep in a function, such as a for loop or switch and need to allocate and release some resource. The resource has to be released immediately, instead of when the function returns.

It would be tedious to split off a tiny piece of code into a separate function.

Here’s a slight variation on the previous example. This time Close() requires some extra, time-consuming, code at the end. You don’t want to delay unlocking the mutex until d.SomethingElse() has finished, so you can’t use defer.

func (d *Destination) Close(name string) {
    d.mu.Lock()
    d.connection.Close()
    d.mu.Unlock()
	
   d.SomethingElse()  // time-consuming code
}

This has the same problems as mentioned above, but is fixed fairly easily by adding a new named function or by using a function literal like this:

func (d *Destination) Close(name string) {
    func() {
        d.mu.Lock()
        defer d.mu.Unlock()
        d.connection.Close()
    }()

    d.SomethingElse()
}

I don’t know about you but my functions tend to become overly long. Splitting off smaller functions to enable the use of defer is a good thing.

Defer and Panic

As you are aware, Go has “exceptions” much like the throw, catch mechanism of C++, Java, etc, but uses the keyword panic and the built-in function recover(). This is tightly linked to defer since you can only call recover() with a deferred function.

Recover is easy to use in Go, once you understand it. Here’s a question from a recent quiz I gave at the Sydney Go Meetup. This one catches people out if they are used to C++.

func main() {
    defer func() {
        defer func() {
            fmt.Print(recover())
        }()
        fmt.Print(recover())
        panic("DEF")
    }()
    panic("ABC")
}

What does this print? (Answer: below)

Defer and Return

Another useful (if a little bit tricky) feature of Go is that deferred functions are executed after the function’s return statement is executed. This allows you to change the return values from within the deferred function, but only if you use named return values.

TODO: close example

Just to check that you understand how this works here is another puzzle:

package main

import "fmt"

func main() {
	fmt.Println(f(), g())
}

func f() int {
	var r int
	defer func() {
		r = 42
	}()
	return r
}

func g() (r int) {
	defer func() {
		r = 42
	}()
	return r
}

What does this print? (Answer: below)

The Close Mistake

A common mistake is to assume that closing a file does not have an error.

func save(filename string) (err error) {
    if f, err := os.Create(filename); err != nil
        return err
    }
    defer f.Close()  // WARNING: ignores errors
	if _, err = f.Write(...
    ...

But f.Close() can return an error, since it can write buffered data to disk. You need to check the error return value within the deferred f.Close(). Since the function has a named return value you can set the error return from inside the deferred function.

func save(filename string) (err error) {
    if f, err := os.Create(filename); err != nil
        return err
    }
    defer func() {
        err = f.Close()
    }()
    if _, err = f.Write(...
    ...

When Are Arguments Evaluated?

One pitfall that often catches new gophers is the assumption that arguments are evaluated when the deferred function is called. Here is another puzzle for you:

func f(i int) {
    fmt.Println("value =", i)
}

func main() {
    a := 1
    defer f(a)
    a = 42
}

What does this print? (Answer: below)

In order use the value at the time the deferred function is called you need to capture the value like this:

func main() {
	a := 1
	defer func() {
		fmt.Println("value =", a) // value = 42  
	}()
	a = 42
}

Returning a Deferred Function

Another pattern you may see is calling a function that returns a function to be deferred. Here is an example:

func g() func() {
	fmt.Println("In g")
	return func() {
		fmt.Println("In deferred func")
	}
}

func main(t *testing.T) {
	defer g()()
	fmt.Println("In main")
}

Which will print:

In g
In main
In deferred func

Conclusion

There’s just one point to this post: always use defer when you need to clean-up or free resources. Using defer will make your code robust to future changes and safe in the presence of (recovered) panics.

As we saw above this is true even in the simplest cases. You might be tempted in more complex cases (where using defer necessitates creating a new function) to skip it. You should take a little bit more time to refactor the code; this has the added benefit that it encourages smaller functions.

The main thing I like about defer is that you can put the code that frees a resource immediately after the code that allocates it. This proximity makes it easier to understand and check what the code is doing.

Puzzle Answers

ABCDEF

0 42

value = 1