Go 1.22 Released

Go 1.22

It’s time to look closely at Go 1.22 now that it is officially released. There is a lot of interest in addressing a gotcha that seems to affect most new Gophers - see capture of loop variables.

Apart from some cute changes to for loops, there are some significant additions to the standard library and some nice optimizations, including some for PGO.

But a major improvement that seems to have slipped under the radar is that the Go execution tracer has been overhauled (dare I say fixed :). Most significantly (at least to me) is that issues with traces not being able to be displayed due to time stamps out of order no longer occur.

The execution tracer is an amazing tool (unique to Go) which allows you to understand in detail what your goroutines are doing and how they are interacting with each other, the garbage collector, OS, etc.

Background

General

C# Capture of loop variables

When I first tried lambdas in C# (about 20 years ago), I discovered something unexpected. When you create lambdas within a loop which captures the loop variable they all capture the same variable. After the loop has completed all the captured variables have the same value – the final value of the loop variable. What I, and apparently others, expected was the captured variable would reflect the value of the loop variable at the time of capture.

The C# behaviour makes sense because there is only one loop variable and variables captured in lambdas are references to the original variable. However, it seemed to me that a special case could be made for loop variables - ie if captured then a copy of the current loop variable value could be captured.

I haven’t used C# in yonks, so I am not sure if this has been addressed, but I suspect not.

---

Go

Go Capture of loop variables

When I first captured a loop variable in a Go closure, I was alert to the issue with the capture of loop variables from my prior experience with C# (see above).

But I was a bit surprised that the Go designers had not learnt from use of other languages, like C#, and captured a copy of the loop variable. I assumed the decision was made for reasons such as efficiency.

I suspect it may have been done like that as a learning tool. Once you encounter the gotcha, you learn, or recall, that variables are captured by reference!

This problem seems to affect Go newbies more than users of other languages. I suspect that this is because of how much easier it is to use concurrency – I think all Gophers have started a new goroutine in a loop something like this:

   for i := range s {
      go func() {
         println(i)
      }()
   }

The output of this loop is undefined as Go does not say when each new goroutine, created in the loop, will run. Depending on when they run it might print 0 1 2 3… or 2 5 5 5…. Usually, the loop terminates before any goroutine runs, and it just prints the final value of i.

Parallel Tests

I encountered this problem in some table-driven tests. It is quite easy to capture a loop variable using “sub-tests” (ie, calling testing.T.Run()).

I was very surprised when parallelising tests (to make them run faster) that many started to fail. The closures being passed to t.Run() were capturing a loop variable. As an example - see TestSplit() in Introducing sub tests.

These tests worked fine for years since they ran serially, but just changing the code to call t.Parallel() introduced a race condition. The same thing apparently happened in many Go standard library tests - see Go Playground example

Finding the Problem (before Go 1.22)

A check for this problem was explicitly added to Go vet a few years ago. See .

$ go vet
...
.\main.go:5:23: loop variable i captured by func literal

This will also be detected at run-time using the race detector, since there is a data race on the loop variable. (Of course, it’s better to catch the problem before run-time.)

Fixing the Problem (before Go 1.22)

The problem is that the func (closure) running in the goroutine captures i as a reference to the original i. This can be addressed by taking a copy of i - conventionally done using a local variable of the same name.

   for i := range s {
      go func() {
         i := i // local i captures "outer" i (loop variable)
         println(i)
      }() 
   }

Another way be to take a copy of i by passing it as a parameter to the closure.

   for i := range s {
      go func(arg int) {
         println(arg)
      }(i)
   }

---

Loops

There have been two enhancements (plus a preview) to for loops in Go 1.22.

Capture of Loop Variables

A common mistake for new Gophers is to capture a loop variable in a goroutine, expecting it to retain the value at the time of capture.

Despite numerous explanations in popular blogs saying so, this has nothing to do with goroutines, though often the go keyword is involved - as in the example in Background/Go above.

For example, this code demonstrates the issue without involving goroutines.

   var s []func()
   for i := 0; i < 3; i++ {
      s = append(s, func() { println(i) })
   }
   for _, f := range s {
      f()
   }

Go Vet

The go vet tool will sometimes warn you about this problem. See Background/Go/Finding the Problem (before Go 1.22) above. (If you haven’t heard of go vet see this excellent post Go: Vet Command Is More Powerful Than You Think)

Warning go vet does not warn in many cases (such as the code just above). I think this is to avoid giving too many false positives. Moreover, since there is no data race the race detector will also not signal a problem.

Go 1.22 Change

Go 1.22 changes the behaviour – the captured variable is a copy of the loop variable. In others words before Go 1.22 the above code would print the final value of the loop variable.

3
3
3

but in Go 1.22 it prints

0
1
2

You can try this in the Go Playground - click the Go version drop-down to see the difference between Go 1.21 and Go 1.22.

Note This change affects any use of the address of the loop variable – not only closures (which capture by taking the address), but also use of the address & operator.

Backward Compatibility

This different behaviour means that Go 1.22 is not backward compatible.

Go is renowned for its backwards compatibility so how can this be? After all, it’s possible that there is code out there that depends on it for correct (or expected) behaviour. Well, apparently production code is generally not affected - see How often does the change break real programs?

But, just in case, the Go 1.22 compiler retains backward compatibility - at least, until you update the go version number in go.mod to go 1.22 or later.

Of course, sooner or later you might need to use Go 1.22, whence you will have to change your code or verify that the changed behaviour has no effect (or even makes your code better :). You can find all the places in your code that are affected using the -gcflags=all=-d=loopvar=2 command line option. See Finding code affected by the change

Range Over Integers

Go 1.22 also adds loop over integers. So instead of this very common sort of code:

    for i := 0; i < 10; i++ { ... }

you can do this:

    for i := range 10 { ... }

Unless I am missing something, I can’t see much difference. (I’ll try to remember to use it, but after 4 decades of C, C++, C# and Go, I think my loop code is too ingrained :)

Range Over Function (preview)

The ability to range over a function is more useful, but it is not part of the language (yet). It is included in 1.22 as a preview and I suspect that it will change significantly before it becomes part of the language proper.

It seems that the Go team are trying to address Go’s inability to create efficient user-defined iterators. This has become more urgent with the advent of generics, since they allow containers to be created that could benefit from iterators. However, I suspect that efficient iterators will eventually find there way into the language through Coroutines - see Coroutines for Go.

Runtime

Execution Tracer

In the past I’ve had a love/hate relationship with the execution tracer. For simple things it does an amazing job but for some real production issues I found collecting and viewing traces slow and confusing. Large traces had to be split for viewing.

Worst of all, I would get an error when I tried to view complex (ie, useful) traces.

I am very happy to report that these problems have been addressed. Gathering traces has less impact on performance. Larger traces are easier to be view and less likely to need splitting. The time stamps out of order error never happens anymore.

There are also some other improvements as a result of the complete overhaul. You can now stream traces as they are generated, rather than having to save a complete trace to file.

One enhancement that I am keen to try is that these low-level traces can coordinate with “high-level” tracing tools such as Open Telemetry.

Standard Library

Random Numbers

There is a new math/rand/v2 package intended to replace math/rand. It’s nice to have a generic versions of rand.IntN() that works with any integer type. But for serious randomness you should probably be using crypto/rand.

HTTP Routing

There have been many problems with the standard library HTTP muxer (aka router), which may be why there seem to be hundreds of open-source alternatives. However, I have modest requirements (and mostly use GraphQL), so usually use the standard library net/http package.

It is nice that it has the following new features.

• method matching - previously you had to manually check that a handler was called for the correct HTTP method (PUT, POST, etc)
• wildcards/variables in paths - provides greater flexibility (as in almost all open-source routers)

From my understanding there are still issues with performance (though not significant to me) There are also some security issues if you use the standard library, which can be addressed with a bit of work.

Conclusion

With these improvements I think the Go Execution Trace will be used much more, at least by me. I have already used it to verify the source of a very obscure performance issue in production. I hope to write something soon about using the execution tracer as it is such an amazingly useful tool.

I haven’t mentioned the numerous other changes and fixes to the tools, standard library, etc. See Release Notes for details.