This post is about the experience we had with the omitempty JSON tag while constructing JSON in Golang. omitempty ignores the null value of datatypes. This could cause problems when the expected value for an attribute is the null value.

I started experiencing this problem when I was adding a new API. Our distribution platform archives movie content after N days. I was implementing an API that lists movies from the archives along with information like the size of each movie and the time taken for retrieving every movie from the archives.

Model:

This is the struct definition we use to list movies.

1
2
3
4
5
6
7
8
9
type Response struct {
  Movies []Movie
}

type Movie struct {
  ID                    string
  AvailabilityInMinutes int    `json:"availabilityInMinutes,omitempty"`
  Size                  uint64 `json:"size,omitempty"`
}

Let us consider that this API returns 2 movies of which the first movie is available immediately while the other movie is not present in the archives.

Data Initialisation

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
movies := []Movie{
  {
    ID: "3f00c091-b29d-4b99-8136-5cb65c987250", 
    AvailabilityInMinutes: 0,
    Size: 54626741
  },
  {
    ID: "04ea246b-7e7a-46a4-ba56-867117a90610", 
  }
}

moviesJSON, err := json.Marshal(movies)
if err != nil {
  handleErr(err)
}

Ideally, for the first movie, the API should list all the properties. The API should only list the id property for the second movie since the system doesn’t have any information about the movie.

Expected Response:

1
2
3
4
5
6
7
8
9
10
11
12
{
  "movies": [
    {
      "id": "3f00c091-b29d-4b99-8136-5cb65c987250",
      "availabilityInMinutes": "0",
      "size": "54626741"
    },
    {
      "id": "04ea246b-7e7a-46a4-ba56-867117a90610"
    }
  ]
}

The Problem

However, the response that we got was:

1
2
3
4
5
6
7
8
9
10
11
{
  "movies": [
    {
      "id": "3f00c091-b29d-4b99-8136-5cb65c987250",
      "size": "54626741"
    },
    {
      "id": "04ea246b-7e7a-46a4-ba56-867117a90610"
    }
  ]
}

The availability field for both movies is missing! For the first movie,availabilityInMinutes has been omitted since it holds 0 which is the default value/empty value for int datatype. For the second movie,availabilityInMinutes has been omitted since the attribute does not hold any value. If we remove the omitempty tag in that field, then the second movie that is not present in the system will also have availabilityInMinutes set to 0 which is incorrect.

After discussing with the team, we decided to use the pointer type.

The solution:

The fix is to use the pointer to the primitive type as the pointers have nil as the zero values.

Updated Model

1
2
3
4
5
type Movie struct {
  ID                    string
  AvailabilityInMinutes *int   `json:"availabilityInMinutes,omitempty"`
  Size                  uint64 `json:"size,omitempty"`
}

Data

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
movie1Availability := 371367181
movies := []Movie{
  {
    ID: "3f00c091-b29d-4b99-8136-5cb65c987250",
    AvailabilityInMinutes: new(int),
    Size: 54626741
  },
  {
    ID: "04ea246b-7e7a-46a4-ba56-867117a90610",
  }
}

moviesJSON, err := json.Marshal(movies)
if err != nil {
  handleErr(err)
}

On marshaling the above data, we get the following JSON:

1
2
3
4
5
6
7
8
9
10
11
12
{
  "movies": [
    {
      "id": "3f00c091-b29d-4b99-8136-5cb65c987250",
      "availabilityInMinutes": 0,
      "size": "54626741"
    },
    {
      "id": "04ea246b-7e7a-46a4-ba56-867117a90610"
    }
  ]
}

And the problem was solved!

TL;DR

Using the pointers (whose zero values are nil) in place of the primitive types solves the problem.