Go Slices Demystified

Slices are one of those types in Go that take a little bit of time and hands on experience to really wrap your head around. There is a lot of material out there that explain slices, but I'm going to take a slightly different approach. We'll start with clarifying some potential misconceptions with the mechanics of slices, and prove out their implementation with executable Go code.

Slices are value types

In practice, while it may appear that slices are pointer types, in actuality they are value types.

package main

func main() {
  var myslice []int
  var mypointer uintptr
  
  fmt.Printf("slice size: %v | pointer size: %v", unsafe.Sizeof(myslice), unsafe.Sizeof(mypointer))
}

Output

$ go run main.go
slice size: 24 | pointer size: 8

A uintptr, at least on the host machine, is 8 bytes. The slice, []int is three times the size, 24 bytes.

This is due to the fact that a slice is a value type, specifically a struct, that contains three fields: Data, Len, and Cap.

See the SliceHeader documentation in the reflect package for more information. Alternatively, you can look at slice.go from the Go runtime.

This means that when you pass a slice from one function to another, you are copying an entire struct, not just a pointer.

As a bonus, since the map type often gets lumped into the same category as slices when talking about pointer types, a map in Go actually is a pointer type.

package main

func main() {
  var mymap map[int]int
  var mypointer uintptr
  
  fmt.Printf("map size: %v | pointer size: %v", unsafe.Sizeof(mymap), unsafe.Sizeof(mypointer))
}

Output

$ go run main.go
map size: 8 | pointer size: 8

Slices point to arrays

While arrays in Go are immutable, slices are not. They do, however, just reference an array under the hood. If you clicked on either of the above references discussing slice internals, you probably saw this already.

You'll notice that when it comes to growing in size, the behavior of a slice is what you would expect from an array. Hopefully you remember your lectures from data structures!

package main

import (
  "fmt"
)

func main() {
  myslice := []string{"0"}

  // &myslice[0] is used here because myslice itself references the struct of the slice
  // slice[0] is the 0th element of the underlying array for the slice
  // if the address of slice[0] changes, we know that the underlying array has been modified
  fmt.Printf("len %v | cap %v | array addr %p | slice addr %p\n", len(myslice), cap(myslice), &myslice[0], &myslice)
  
  myslice = append(myslice, "1")
  fmt.Printf("len %v | cap %v | array addr %p | slice addr %p\n", len(myslice), cap(myslice), &myslice[0], &myslice)
  
  myslice = append(myslice, "2")
  fmt.Printf("len %v | cap %v | array addr %p | slice addr %p\n", len(myslice), cap(myslice), &myslice[0], &myslice)
  
  myslice = append(myslice, "3")
  fmt.Printf("len %v | cap %v | array addr %p | slice addr %p\n", len(myslice), cap(myslice), &myslice[0], &myslice)
}
$ go run main.go
len 1 | cap 1 | array addr 0xc0000581c0 | slice addr 0xc000062420
len 2 | cap 2 | array addr 0xc000062460 | slice addr 0xc000062420
len 3 | cap 4 | array addr 0xc000068100 | slice addr 0xc000062420
len 4 | cap 4 | array addr 0xc000068100 | slice addr 0xc000062420

From the execution above, we can see that the initial creation of the slice sets both the capacity and length to 1.

When we perform an append() operation to add another value to the slice, we see that the capacity and length are incremented to 2.

You'll notice however, that the address changed after the append. This is because the underlying array reached capacity, and in order to be able to store the new value, needed to be destroyed and recreated with more space.

When capacity is reached, the internal implementation is that the original array should be destroyed, and a new array with double the capacity of the old array should be created. The original data will then be copied over to the new array.

We can see all of these changes, because append returns the slice with the new capacity and length values. Potentially a new backing array if a destroy was required.

But again, slices reference arrays under the hood, slices are just a slice of an array and don't contain the elements themselves. What happens if we don't look at the value that is returned from the append?

  ...
  _ = append(myslice, "5")
  fmt.Printf("len %v | cap %v | array addr %p | slice addr %p\n", len(myslice), cap(myslice), &myslice[0], &myslice)
$ go run main.go
len 4 | cap 4 | array addr 0xc000068100 | slice addr 0xc000062420

Everything stays the same! Which may not be shocking, but the interesting part is that the underlying array did change. And we can prove that.

package main

import (
  "fmt"
  "reflect"
  "unsafe"
)

func main() {
  myslice := []string{"0"}
  fmt.Printf("len %v | cap %v | array addr %p | slice addr %p\n", len(myslice), cap(myslice), &myslice[0], &myslice)
  
  myslice = append(myslice, "1")
  fmt.Printf("len %v | cap %v | array addr %p | slice addr %p\n", len(myslice), cap(myslice), &myslice[0], &myslice)
  
  myslice = append(myslice, "2")
  fmt.Printf("len %v | cap %v | array addr %p | slice addr %p\n", len(myslice), cap(myslice), &myslice[0], &myslice)
  
  // Call the append() function, but do not update the slice to reflect the changes
  _ = append(myslice, "3")
  fmt.Printf("len %v | cap %v | array addr %p | slice addr %p\n", len(myslice), cap(myslice), &myslice[0], &myslice)

  // Magic time! Using myslice, its header which contains .Data (the underlying array), .Len, and .Cap
  sliceHeader := (*reflect.SliceHeader)(unsafe.Pointer(&myslice))

  // Using the .Data field, cast it into a pointer to an array of strings, its native type
  underlyingArray := (*[4]string)(unsafe.Pointer(sliceHeader.Data))
  
  fmt.Printf("len %v | cap %v | array addr %p (unchanged) | val %v (last appended value)\n", len(underlyingArray), cap(underlyingArray), &underlyingArray[0], underlyingArray[3])
}
$ go run main.go
len 1 | cap 1 | array addr 0xc0000581c0 | slice addr 0xc000064420
len 2 | cap 2 | array addr 0xc000064460 | slice addr 0xc000064420
len 3 | cap 4 | array addr 0xc000068100 | slice addr 0xc000064420
len 3 | cap 4 | array addr 0xc000068100 | slice addr 0xc000064420
len 4 | cap 4 | array addr 0xc000068100 (unchanged) | val 3 (last appended value)

You can see from the output that the while the slice has length 3, we referenced the 4th element in the array (at index 3), and got the last value we appended. The slice remained unchanged, but we can clearly see something still happened.

To accomplish this, we took a pointer to our slice, myslice and casted it into a representation of a SliceHeader from the reflect package. We can then take that struct, and get the underlying array via the .Data property by casting it to an array of strings.

If we so desired, we could update the struct after the fact by increasing the size of the slice, allowing it to access more of the array.

// [:4] means allow the slice to see the 0th index of the underlying array up until the 3rd index.
// This allows us to call myslice[3], which before, caused an out of bounds error
myslice = myslice[:4]

TL;DR I just want to regurgitate stuff

  1. Slices are value types. They are not pointers
  2. The slice type is a struct which has three private fields: array, len, and cap
  3. Slices hold a reference to an array, so it's possible that multiple slices reference the same array