Embedding Files in your Go Lang binary

We talked about how I built https://status.barelyhuman.dev and what I was doing to handle adding the html data into the final binary so that the hosting provider could render the html from a single binary.

You can read that post here

The previous solution works and was quite easy to do but then it obviously took away my speed since there's no syntax highlighting for css or html as they are all just go strings now. Plus, to format the html or css I have to move them to a separate file format it there and paste it back here which is really not something I wish to do if I have to maintain the repo for a long time.

So i started looking up ways and while I knew about go embed I was waiting for embed filesystem(FS) to get a little more stable and then I forgot about it and never checked the pull requests that were to be merged for embed FS. I happened to randomly check it two days back and so I'm now going to talk about how embed works and how status is using it to handle the HTML and style templates.

html/template

This assumes that you've worked with html/templates in go before and we build up on that. The basic html/template code flow is something like this

  1. Parse the templates, either as files , glob, or the entire file system with a certain pattern.
  2. Execute the template with the needed variable data for the actual compilation to be done
  3. Write the compiled template to an io.Writer instance, in our case, the http socket

embed

embed provides two ways to go about embedding files or other static data

  1. Read the file into a byte slice
  2. Read the file into the embed filesystem

If we were to use the first one the code would look a little something like

package extras

import (
	_"embed"
	"html/template"
)

//go:embed templates/home.html
var homeHTML []byte

the modifier you need to look at is go:embed this allows you to put a path based on the current file, you cannot use files from folder outside your current .go file , which is the current limitation of pattern matching but I really hope changes in the future.

The compiler then reads and embeds the data into the variable homeHTML and then you can continue using the file like it's already read. but in out case we have more than one template and you can add that to each variable , just know that the go:embed has to be on the top level and cannot be inside a function (at least, when I'm typing this.)

The other way involves using the embed FS which is basically going to do the same thing but instead creates an in-binary filesystem that you can read through. This is built on the existing go FS interface so you can use the embedFS wherever the FS interface is supported which the html/template library does and we are going to use that since I don't want to manually have to go and add the embed since I am parsing all templates anyway.

The code for the same would look something like this

package extras

import (
	"embed"
	"html/template"
)

//go:embed templates/*.html
var embedFS embed.FS


// GetTemplates - parse and get all templates
func GetTemplates() (*template.Template, error) {
	allTemplates, err := template.ParseFS(embedFS, "templates/*.html","styles/styles.html")
	if err != nil {
		return nil, err
	}
	return allTemplates, nil
}

The line of focus is again the go:embed comment but also focus on the next line this time, we now have a variable that is of the type embed.FS which as I mentioned before is an implementation of the filesystem interface that go already has for other io interfaces and since the html/template allows you to parse based on filesystem and from the 2nd param you can add in patterns which is of the type (...strings) aka infinite patterns on the FS and everything gets parsed into allTemplates.

At this point, all I do is call GetTemplates from wherever I need them, and follow the normal template code flow

  1. Compile them with the required variable data
  2. Pass them to the io.Writer interface, in our case the http socket

and we are basically done, as a user, this won't create a difference on what you see when you open status but as a dev, this made it a lot more easier for me to handle changes in the html and style files.