Zero allocated byte to string conversion

Go has a built-in function for converting a slice of bytes to a string, but it involves allocating a new string. If you want to avoid this allocation, you can use a trick that involves the unsafe package.

I recently learned about this trick from a tweet in prometheus repository and followed the discussion on Github. It turns out that this trick was originally used in the strings package, and it works by using type coercion to convert the memory address of the slice to a string pointer, and then dereferencing that pointer to get the string value. Here is the code:

func ByteSlice2String(b []byte) string {
    if len(b) == 0 {
        return ""
    }

    return *((*string)(unsafe.Pointer(&b)))
}

Here's how it works:

1. unsafe.Pointer(&bs) takes the memory address of the slice bs and converts it to an unsafe.Pointer. An unsafe.Pointer is a special type in the unsafe package that can hold the address of any type.
2. (*string)(unsafe.Pointer(&bs)) converts the unsafe.Pointer to a string pointer by performing type coercion. This creates a new string pointer that points to the same memory address as the slice bs.
3. *(*string)(unsafe.Pointer(&bs)) dereferences the string pointer to get the string value that it points to.

Prove it

This experiment shows the benchmark between the string(bs) and our ByteSlice2String function

var (
    someBytes = []byte(`hello`)
    str       string
)

func BenchmarkByte2String(b *testing.B) {
    b.Run("BuiltIn", func(b *testing.B) {
        for i := 0; i < b.N; i++ {
            str = BuiltIn(someBytes)
        }
        b.ReportAllocs()
    })

    b.Run("Byte2Slice", func(b *testing.B) {
        for i := 0; i < b.N; i++ {
            str = ByteSlice2String(someBytes)
        }
        b.ReportAllocs()
    })
}

Benchmark result shows that the function ByteSlice2String has zero allocation memory.

goos: darwin
goarch: amd64
pkg: github.com/ntk12/temp
cpu: Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
BenchmarkBuildString
BenchmarkBuildString/BuiltIn
BenchmarkBuildString/BuiltIn-16          75249337         15.88 ns/op        5 B/op        1 allocs/op
BenchmarkBuildString/Byte2Slice
BenchmarkBuildString/Byte2Slice-16       324706618          3.652 ns/op        0 B/op        0 allocs/op
PASS
ok   github.com/ntk12/temp 4.461s

When this trick goes wrong

Using the unsafe package and type coercion to convert a slice of bytes to a string can lead to unpredictable behavior and can compromise the safety guarantees provided by Go. It may also produce incorrect results if the slice of bytes does not have the correct memory layout or encoding

The new way in go 1.20

Go 1.20 adds SliceData, String, and StringData which supports our cases in the CL. You can use the new function as follows:

func ByteSlice2String(b []byte) string {
    if len(b) == 0 {
        return ""
    }
    return unsafe.String(&b[0], len(b))
}

func String2ByteSlice(s string) []byte {
    return unsafe.Slice(unsafe.StringData(s), len(s))
}