Serving multiple SSL certificates in your Go tests

Over the past few months, I’ve been redesigning and writing Scopify’s SSL monitoring feature from Node to Go. This blog post describes one of the more subtle challenges we came across to help you master it if you find yourself with it too!

Writing a Go client that fetches an SSL certificate isn’t a new problem. A common approach is to use a http.Client. This limits you to just certificates served over HTTPS, when technically anything running TLS can have a certificate. We decided to use the tls package instead.

conn, err := tls.Dial("tcp", url, &t.config)
if err != nil {
    return err
}
defer conn.Close()

cs := conn.ConnectionState()

// First is the entity certificate
// Second is the intermediate certificate (signs the entity)
switch len(cs.PeerCertificates) {
case 0:
    return errors.New("entity certificate not found")
case 1:
    return errors.New("intermediate certificate not found")
}
fmt.Println("Entity: ", cs.PeerCertificates[0].Subject.CommonName)
fmt.Println("Intermediate: ", cs.PeerCertificates[1].Subject.CommonName)

Running this for url = "octulus.com:443", we get:

Entity: *.octulus.com
Intermediate: Sectigo RSA Domain Validation Secure Server CA

The important thing to note here is that we receive both the entity and intermediate certificate.

Testing, testing, testing

I needed my tests to be able to:

• Spoof a server that spits out a certificate for each link in the SSL chain (entity and intermediate; in this case I didn’t care about the root)
• Generate certificates programmatically (i.e. OpenSSL… 👋)

For a tutorial on generating a spoofed certificate and serving it, look no further than this article by Shane Utt’s. We’ll try it first.

We’ve generated two certificates:

• Our entity certificate, which is signed by…
• Our certificate authority (CA), which is signed by itself.

var (
	// Intermediate CA certificate
	intermediateCA = x509.Certificate{
		SerialNumber: big.NewInt(2020),
		Subject: pkix.Name{
			CommonName:    "Intermediate Cert Authors",
			Organization:  []string{"IntermediateCerts Ltd."},
			Country:       []string{"UK"},
			Province:      []string{""},
			Locality:      []string{"London"},
			StreetAddress: []string{"The World's End, Finsbury Park"},
			PostalCode:    []string{"N4 3EF"},
		},
		Issuer:                testRootCA.Subject,
		SignatureAlgorithm:    x509.SHA256WithRSA,
		PublicKeyAlgorithm:    x509.RSA,
		Version:               3,
		IPAddresses:           []net.IP{net.IPv4(127, 0, 0, 1), net.IPv6loopback},
		NotBefore:             time.Date(2020, 06, 25, 0, 0, 0, 0, time.UTC),
		NotAfter:              time.Date(2021, 06, 25, 0, 0, 0, 0, time.UTC),
		IsCA:                  true,
		ExtKeyUsage:           []x509.ExtKeyUsage{x509.ExtKeyUsageClientAuth, x509.ExtKeyUsageServerAuth},
		KeyUsage:              x509.KeyUsageDigitalSignature | x509.KeyUsageCertSign,
		BasicConstraintsValid: true,
	}

	// Bog standard SSL entity certificate (bottom of the chain)
	entityCert = x509.Certificate{
		SerialNumber: big.NewInt(2019),
		Subject: pkix.Name{
			CommonName:    " Scopify",
			Organization:  []string{"TrafficCake Ltd."},
			Country:       []string{"UK"},
			Province:      []string{""},
			Locality:      []string{"London"},
			StreetAddress: []string{"The Faltering Fullback, Finsbury Park"},
			PostalCode:    []string{"N4 3HB"},
		},
		Issuer:             testIntermediateCA.Subject,
		SignatureAlgorithm: x509.SHA256WithRSA,
		PublicKeyAlgorithm: x509.RSA,
		Version:            3,
		IPAddresses:        []net.IP{net.IPv4(127, 0, 0, 1), net.IPv6loopback},
		NotBefore:          time.Date(2020, 06, 25, 0, 0, 0, 0, time.UTC),
		NotAfter:           time.Date(2021, 06, 25, 0, 0, 0, 0, time.UTC),
		SubjectKeyId:       []byte{1, 2, 3, 4, 6},
		ExtKeyUsage:        []x509.ExtKeyUsage{x509.ExtKeyUsageClientAuth, x509.ExtKeyUsageServerAuth},
		KeyUsage:           x509.KeyUsageDigitalSignature,
	}
)

// Get a tls.Certificate
serverCert, err := certsetup()
if err != nil {
    panic(err)
}

// Set up the httptest.Server using our certificate signed by our CA
srv := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {fmt.Fprintln(w, "success!")}))
srv.TLS = &tls.Config{
    Certificates: []tls.Certificate{serverCert},
}

srv.StartTLS()
defer srv.Close()

Easy. Let’s TLS dial as we did earlier — surely it will return both of these certificates, right?

err: intermediate certificate not found

Whaaaaaat?! Turns out our server didn’t serve two certificates like it would in the real world. The issue is the entity certificate is only signed by the CA; the server doesn’t actually return the CA’s certificate.

So let’s fix this. Straight away, you notice the TLS config’s certificate attribute only includes the one certificate — just add the CA certificate to it, right?

The naive (wrong) solution

// Get two tls.Certificate:
// - Entity (our server's subject)
// - Intermediate (the certificate for the CA that signs the entity)
entityCert, intermediateCert, err := certsetup()
if err != nil {
    panic(err)
}

// Set up the httptest.Server using our certificate signed by our CA
srv := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {fmt.Fprintln(w, "success!")}))
srv.TLS = &tls.Config{
    Certificates: []tls.Certificate{entityCert, intermediateCert},
 }

srv.StartTLS()
defer srv.Close()

Oh, how I wished it were this simple.

You’d be forgiven for thinking the Certificates attribute is a slice of certificates to serve to a client. Spoiler: It’s not.

It’s actually a series of certificates (chains) to serve to the client; the first certificate compatible with the client’s requirements is used. So with our new ‘solution’, we’re still just serving the first certificate, since it meets the client’s requirements.

The (right) solution

Create a certificate chain as a tls.Certificate struct and use this in the Certificates slice.

Let’s run through this. We have the certificates:

var (
	// Root certificate authority
	rootCA = x509.Certificate{
		SerialNumber: big.NewInt(2020),
		Subject: pkix.Name{
			CommonName:    "Root Cert Authors",
			Organization:  []string{"RootCerts Ltd."},
			Country:       []string{"CA"},
			Province:      []string{""},
			Locality:      []string{"Vancouver"},
			StreetAddress: []string{"Cosy Inn Cafe, Dunbar Street"},
			PostalCode:    []string{"V6S 2G4"},
		},
		SignatureAlgorithm:    x509.SHA256WithRSA,
		PublicKeyAlgorithm:    x509.RSA,
		Version:               3,
		IPAddresses:           []net.IP{net.IPv4(127, 0, 0, 1), net.IPv6loopback},
		NotBefore:             time.Date(2020, 06, 25, 0, 0, 0, 0, time.UTC),
		NotAfter:              time.Date(2021, 06, 25, 0, 0, 0, 0, time.UTC),
		IsCA:                  true,
		ExtKeyUsage:           []x509.ExtKeyUsage{x509.ExtKeyUsageClientAuth, x509.ExtKeyUsageServerAuth},
		KeyUsage:              x509.KeyUsageDigitalSignature | x509.KeyUsageCertSign,
		BasicConstraintsValid: true,
	}

	// Intermediate CA certificate
	intermediateCA = x509.Certificate{
		SerialNumber: big.NewInt(2020),
		Subject: pkix.Name{
			CommonName:    "Intermediate Cert Authors",
			Organization:  []string{"IntermediateCerts Ltd."},
			Country:       []string{"UK"},
			Province:      []string{""},
			Locality:      []string{"London"},
			StreetAddress: []string{"The World's End, Finsbury Park"},
			PostalCode:    []string{"N4 3EF"},
		},
		Issuer:                testRootCA.Subject,
		SignatureAlgorithm:    x509.SHA256WithRSA,
		PublicKeyAlgorithm:    x509.RSA,
		Version:               3,
		IPAddresses:           []net.IP{net.IPv4(127, 0, 0, 1), net.IPv6loopback},
		NotBefore:             time.Date(2020, 06, 25, 0, 0, 0, 0, time.UTC),
		NotAfter:              time.Date(2021, 06, 25, 0, 0, 0, 0, time.UTC),
		IsCA:                  true,
		ExtKeyUsage:           []x509.ExtKeyUsage{x509.ExtKeyUsageClientAuth, x509.ExtKeyUsageServerAuth},
		KeyUsage:              x509.KeyUsageDigitalSignature | x509.KeyUsageCertSign,
		BasicConstraintsValid: true,
	}

	// Bog standard SSL entity certificate (bottom of the chain)
	entityCert = x509.Certificate{
		SerialNumber: big.NewInt(2019),
		Subject: pkix.Name{
			CommonName:    " Scopify",
			Organization:  []string{"TrafficCake Ltd."},
			Country:       []string{"UK"},
			Province:      []string{""},
			Locality:      []string{"London"},
			StreetAddress: []string{"The Faltering Fullback, Finsbury Park"},
			PostalCode:    []string{"N4 3HB"},
		},
		Issuer:             testIntermediateCA.Subject,
		SignatureAlgorithm: x509.SHA256WithRSA,
		PublicKeyAlgorithm: x509.RSA,
		Version:            3,
		IPAddresses:        []net.IP{net.IPv4(127, 0, 0, 1), net.IPv6loopback},
		NotBefore:          time.Date(2020, 06, 25, 0, 0, 0, 0, time.UTC),
		NotAfter:           time.Date(2021, 06, 25, 0, 0, 0, 0, time.UTC),
		SubjectKeyId:       []byte{1, 2, 3, 4, 6},
		ExtKeyUsage:        []x509.ExtKeyUsage{x509.ExtKeyUsageClientAuth, x509.ExtKeyUsageServerAuth},
		KeyUsage:           x509.KeyUsageDigitalSignature,
	}
)

NOTE: This isn’t necessary, but for completeness, I’ve added a root certificate to sign our intermediate. It’s a bit more realistic, as we’re not signing the intermediate certificate with itself.

Create our intermediate cert

We want to create a private and public key for the intermediate certificate, have it signed by the root CA and then PEM encode it.

// Create our private and public key for intermediateCA
interCAPrivKey, err := rsa.GenerateKey(rand.Reader, 4096)
if err != nil {
    return tls.Certificate{}, err
}

// Create the intermediate CA certificate
caBytes, err := x509.CreateCertificate(rand.Reader, &cfg.intermediateCA, &cfg.rootCA, &interCAPrivKey.PublicKey, interCAPrivKey)
if err != nil {
    return tls.Certificate{}, err
}

// PEM encode the certificate and private key
interCAPEM := new(bytes.Buffer)
pem.Encode(interCAPEM, &pem.Block{
    Type:  "CERTIFICATE",
    Bytes: caBytes,
})

interCAPrivKeyPEM := new(bytes.Buffer)
pem.Encode(interCAPrivKeyPEM, &pem.Block{
    Type:  "RSA PRIVATE KEY",
    Bytes: x509.MarshalPKCS1PrivateKey(interCAPrivKey),
})

Create our entity cert

Let’s do the same with our entity cert.

certPrivKey, err := rsa.GenerateKey(rand.Reader, 4096)
if err != nil {
    return tls.Certificate{}, err
}

// Create entity certificate, signed by intermediateCA
certBytes, err := x509.CreateCertificate(rand.Reader, &cfg.entityCert, &cfg.intermediateCA, &certPrivKey.PublicKey, interCAPrivKey)
if err != nil {
    return tls.Certificate{}, err
}

certPEM := new(bytes.Buffer)
pem.Encode(certPEM, &pem.Block{
    Type:  "CERTIFICATE",
    Bytes: certBytes,
})

certPrivKeyPEM := new(bytes.Buffer)
pem.Encode(certPrivKeyPEM, &pem.Block{
    Type:  "RSA PRIVATE KEY",
    Bytes: x509.MarshalPKCS1PrivateKey(certPrivKey),
})

Append the PEM encoding

✨ This is the secret sauce ✨

To serve two certificates, we need to append the two certificates together in one-byte slice, then create our tls.Certificate from this.

var cert []byte
// Concatenate the two certs so they're both served to the client
cert = append(certPEM.Bytes(), interCAPEM.Bytes()...)

serverCert, err := tls.X509KeyPair(cert, certPrivKeyPEM.Bytes())
if err != nil {
    return tls.Certificate{}, err
}

And that’s it! Finally, create the TLS server config and pass it to a httptest server:

cfg := &tls.Config{
    Certificates: []tls.Certificate{serverCert}
}

srv := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {}))
srv.TLS = cfg

srv.StartTLS()
defer srv.Close()

Voila! It’s done.

If you enjoyed this blog, check out our other step-by-step guides like Visual Studio Code shortcuts!