Go Slices Demystified

by John Reese

Slices are one of those types in Go that take a little bit of time and hands on experience to really wrap your head around. There is a lot of material out there that explain slices, but I'm going to take a slightly different approach. We'll start with clarifying some potential misconceptions with the mechanics of slices, and prove out their implementation with executable Go code.

Slices are value types

In practice, while it may appear that slices are pointer types, in actuality they are value types.

package main

import (
	"fmt"
	"unsafe"
)

func main() {
	var s []int
	var p uintptr

	fmt.Printf("slice: %v | pointer: %v", unsafe.Sizeof(s), unsafe.Sizeof(p))
}

$ go run main.go
slice: 24 | pointer: 8

A uintptr, at least on the host machine, is 8 bytes. The slice, []int is three times the size, 24 bytes.

This is due to the fact that a slice is a value type, specifically a struct, that contains three fields: Data, Len, and Cap.

See the SliceHeader documentation in the reflect package for more information. Alternatively, you can look at slice.go from the Go runtime.

This means that when you pass a slice from one function to another, you are copying an entire struct, not just a pointer.

As a bonus, since the map type often gets lumped into the same category as slices when talking about pointer types, a map in Go actually is a pointer type.

package main

import (
	"fmt"
	"unsafe"
)

func main() {
  var m map[int]int
  var p uintptr
  
  fmt.Printf("map: %v | pointer: %v", unsafe.Sizeof(m), unsafe.Sizeof(p))
}

$ go run main.go
map: 8 | pointer: 8

Slices point to arrays

You'll notice that when it comes to growing in size, the behavior of a slice is what you would expect from an array. Hopefully you remember your lectures from data structures!

package main

import (
	"fmt"
)

func main() {
	nums := []int{1}
    
	// NOTE: &nums[0] is used here to show the address of 
    // the first element in the array.
    //
    // This is because the variable `nums` (no indexer) references 
    // the struct of the slice.
    //
	// If the address of nums[0] changes, we know that the array that the 
	// slice points to has changed.
	fmt.Printf("len %v | cap %v | addr %p\n", len(nums), cap(nums), &nums[0])

	nums = append(nums, 2)
	fmt.Printf("len %v | cap %v | addr %p\n", len(nums), cap(nums), &nums[0])

	nums = append(nums, 3)
	fmt.Printf("len %v | cap %v | addr %p\n", len(nums), cap(nums), &nums[0])

	nums = append(nums, 4)
	fmt.Printf("len %v | cap %v | addr %p\n", len(nums), cap(nums), &nums[0])

	nums = append(nums, 5)
	fmt.Printf("len %v | cap %v | addr %p\n", len(nums), cap(nums), &nums[0])
}

$ go run main.go
len 1 | cap 1 | addr 0xc00004a1f0
len 2 | cap 2 | addr 0xc0000044a0
len 3 | cap 4 | addr 0xc0000200c0
len 4 | cap 4 | addr 0xc0000200c0
len 5 | cap 8 | addr 0xc00009a000

From the execution above, we can see that the initial creation of the slice sets both the capacity and length to 1.

When we perform an append() operation to add another value to the slice, we see that the capacity and length are incremented to 2.

You'll notice, however, that the address changed after the append. This is because the underlying array reached capacity. In order to be able to store the newly appended value, the slice needs to reference a new, larger array.

To accomplish this, Go will:

  • Create a new array that is double the capacity of the original array
  • Populate the newly created array with the original values
  • Update the pointer on the slice to point to the newly created array

As shown above, we can see all of these changes because append returns the slice with the new capacity and length values (and potentially a new backing array if the original was at capacity).

In the scenario where the slice needs a new backing array, any references to the old array are still valid.

package main

import (
	"fmt"
)

func main() {
	nums := []int{1}

	// Create a copy of the slice that was declared above
	c := nums

	// Sanity check that all of the properties for both slices are identical
	fmt.Printf("len %v | cap %v | addr %p\n", len(nums), cap(nums), &nums[0])
	fmt.Printf("len %v | cap %v | addr %p\n", len(c), cap(c), &c[0])

	// Since nums has a capacity of 1 and we are performing an append 
	// operation, the backing array will need to be replaced with 
	// a larger array
	nums = append(nums, 2)

	// After completing the append operation, nums has a new backing array
	// Note however that newslice still references the old array
	fmt.Printf("len %v | cap %v | addr %p\n", len(nums), cap(nums), &nums[0])
	fmt.Printf("len %v | cap %v | addr %p\n", len(c), cap(c), &c[0])
}

$ go run main.go
len 1 | cap 1 | addr 0xc0000120d0
len 1 | cap 1 | addr 0xc0000120d0
len 2 | cap 2 | addr 0xc000012120
len 1 | cap 1 | addr 0xc0000120d0

So if slices just reference arrays under the hood, and slices are a slice of an array. What happens if we don't look at the value that is returned from the append?

package main

import (
	"fmt"
)

func main() {
	nums := []int{1}
	fmt.Printf("len %v | cap %v | addr %p\n", len(nums), cap(nums), &nums[0])

	nums = append(nums, 2)
	fmt.Printf("len %v | cap %v | addr %p\n", len(nums), cap(nums), &nums[0])

	nums = append(nums, 3)
	fmt.Printf("len %v | cap %v | addr %p\n", len(nums), cap(nums), &nums[0])

	_ = append(nums, 4)
	fmt.Printf("len %v | cap %v | addr %p\n", len(nums), cap(nums), &nums[0])
}

$ go run main.go
len 1 | cap 1 | addr 0xc00004c1e0
len 2 | cap 2 | addr 0xc00005a460
len 3 | cap 4 | addr 0xc000098000
len 3 | cap 4 | addr 0xc000098000

Everything stays the same! Which may not be shocking, but the interesting part is that the underlying array did change. And we can prove that.

package main

import (
	"fmt"
	"reflect"
	"unsafe"
)

func main() {
	nums := []int{1}
	fmt.Printf("len %v | cap %v | addr %p\n", len(nums), cap(nums), &nums[0])

	nums = append(nums, 2)
	fmt.Printf("len %v | cap %v | addr %p\n", len(nums), cap(nums), &nums[0])

	nums = append(nums, 3)
	fmt.Printf("len %v | cap %v | addr %p\n", len(nums), cap(nums), &nums[0])

	// Call the append() function, but DO NOT update the slice 
	// to reflect the changes
	_ = append(nums, 4)
	fmt.Printf("len %v | cap %v | addr %p\n", len(nums), cap(nums), &nums[0])

	// At this point, we already know myslice was unchanged.
	// However, its possible to see the appended value.

	// 1. Get the slice header of myslice (struct with .Data, .Len, and .Cap)
	sliceHeader := (*reflect.SliceHeader)(unsafe.Pointer(&myslice))

	// 2. Using the .Data field in the slice header
	// cast it into a pointer to an array of ints (its actual type).
	array := (*[4]int)(unsafe.Pointer(sliceHeader.Data))

	// REMEMBER: The .Data field in the slice header is the underlying array!
	fmt.Printf("nums len: %v | nums cap: %v\n", len(nums), cap(nums))
	fmt.Printf("last element in the array is: %v", array[3])
}

$ go run main.go
len 1 | cap 1 | addr 0xc0000361f0
len 2 | cap 2 | addr 0xc0000044c0
len 3 | cap 4 | addr 0xc000040080
len 3 | cap 4 | addr 0xc000040080
nums len: 3 | nums cap: 4
last element in the array is: 4

You can see from the output that the while the slice has length 3, we referenced the 4th element in the array (at index 3), and got the last value we appended. The slice remained unchanged, but the underlying array was updated to contain the appended value.

It is worth mentioning that this is a contrived example to showcase some of the internals, and isn't something you'd really see in production code.

In reality, you would just increase the size of the slice so that you were able to interact with more elements in the array. In other words, what the append operation is doing for you when you get its return value.

// nums has a length of 3, so this line 
// would panic with an out of bounds error
fmt.Println(nums[3]) // panic: out of bounds

// Recall, however, that the array itself is still 
// of length 4 and the last appended value is 4

// We can increase the length of nums which lets us 
// access more of the underlying array. 
// This returns a slice with a size and capacity of 4.
nums = nums[:4]

fmt.Println(nums[3]) // prints 4

// NOTE: [:4] means allow the slice to see the 
// 0th index of the underlying array 
// up until the 3rd index.

TL;DR.. I just want to regurgitate stuff

  1. Slices are value types. They are not pointers
  2. The slice type is a struct which has three private fields: array, len, and cap
  3. Slices control what you can and cannot access in its backing array. It's possible to access more of the array by increasing the slice's len property via the append function
  4. Slices hold a reference to an array, so it's possible that multiple slices reference the same array