REST API Error Handling in Go
Handling errors is really important in Go. Errors are first class citizens and there are many different approaches for handling them. Initially I started off basing my error handling almost entirely on a blog post from Rob Pike and created a carve-out from his code to meet my needs. It served me well for a long time, but found over time I wanted a way to easily get a stacktrace of the error, which led me to Dave Cheney’s https://github.com/pkg/errors package. I now use a combination of the two. The implementation below is sourced from my go-api-basic repo, indeed, this post will be folded into its README as well.
Requirements
My requirements for REST API error handling are the following:
- Requests for users who are not properly authenticated should return a
401 Unauthorized
error with aWWW-Authenticate
response header and an empty response body. - Requests for users who are authenticated, but do not have permission to access the resource, should return a
403 Forbidden
error with an empty response body. - All requests which are due to a client error (invalid data, malformed JSON, etc.) should return a
400 Bad Request
and a response body which looks similar to the following:
- All requests which incur errors as a result of an internal server or database error should return a
500 Internal Server Error
and not leak any information about the database or internal systems to the client. These errors should return a response body which looks like the following:
All errors should return a Request-Id
response header with a unique request id that can be used for debugging to find the corresponding error in logs.
Implementation
All errors should be raised using custom errors from the domain/errs package. The three custom errors correspond directly to the requirements above.
Typical Errors
Typically, errors raised throughout go-api-basic are the custom errs.Error
, which looks like:
These errors are raised using the E
function from the domain/errs package. errs.E
is taken from Rob Pike's upspin errors package (but has been changed based on my requirements). The errs.E
function call is variadic and can take several different types to form the custom errs.Error
struct.
Here is a simple example of creating an error
using errs.E
:
When a string is sent, an error will be created using the errors.New
function from github.com/pkg/errors
and added to the Err
element of the struct, which allows retrieval of the error stacktrace later on. In the above example, User
, Kind
, Param
and Code
would all remain unset.
You can set any of these custom errs.Error
fields that you like, for example:
Above, we used errs.Validation
to set the errs.Kind
as Validation
.
Valid error Kind
are:
errs.Code
represents a short code to respond to the client with for error handling based on codes (if you choose to do this) and is any string you want to pass.
errs.Parameter
represents the parameter that is being validated or has problems, etc.
Note in the above example, instead of passing a string and creating a new error inside the
errs.E
function, I am directly passing the error returned by thetime.Parse
function toerrs.E
. The error is then added to theErr
field usingerrors.WithStack
from thegithub.com/pkg/errors
package. This will enable stacktrace retrieval later as well.
There are a few helpers in the errs
package as well, namely the errs.MissingField
function which can be used when validating missing input on a field. This idea comes from this Mat Ryer post and is pretty handy.
Here is an example in practice:
The error message for the above would read title is required
There is also errs.InputUnwanted
which is meant to be used when a field is populated with a value when it is not supposed to be.
Typical Error Flow
As errors created with errs.E
move up the call stack, they can just be returned, like the following:
In the above example, the error is created in the
inner
function -middle
andouter
return the error as is typical in Go.
You can add additional context fields ( errs.Code
, errs.Parameter
, errs.Kind
) as the error moves up the stack, however, I try to add as much context as possible at the point of error origin and only do this in rare cases.
Handler Flow
At the top of the program flow for each service is the app service handler (for example, Server.handleMovieCreate). In this handler, any error returned from any function or method is sent through the errs.HTTPErrorResponse
function along with the http.ResponseWriter
and a zerolog.Logger
.
For example:
errs.HTTPErrorResponse
takes the custom error ( errs.Error
, errs.Unauthenticated
or errs.UnauthorizedError
), writes the response to the given http.ResponseWriter
and logs the error using the given zerolog.Logger
.
return
must be called immediately aftererrs.HTTPErrorResponse
to return the error to the client.
Typical Error Response
For the errs.Error
type, errs.HTTPErrorResponse
writes the HTTP response body as JSON using the errs.ErrResponse
struct.
When the error is returned to the client, the response body JSON looks like the following:
In addition, the error is logged. If zerolog.ErrorStackMarshaler
is set to log error stacks (more about this in a later post), the logger will log the full error stack, which can be super helpful when trying to identify issues.
The error log will look like the following ( I cut off parts of the stack for brevity):
Note:
E
will usually be at the top of the stack as it is where theerrors.New
orerrors.WithStack
functions are being called.
Internal or Database Error Response
There is logic within errs.HTTPErrorResponse
to return a different response body if the errs.Kind
is Internal
or Database
. As per the requirements, we should not leak the error message or any internal stack, etc. when an internal or database error occurs. If an error comes through and is an errs.Error
with either of these error Kind
or is unknown error type in any way, the response will look like the following:
Unauthenticated Errors
The spec for 401 Unauthorized
calls for a WWW-Authenticate
response header along with a realm
. The realm should be set when creating an Unauthenticated error. The errs.NewUnauthenticatedError
function initializes an UnauthenticatedError
.
I generally like to follow the Go idiom for brevity in all things as much as possible, but for
Unauthenticated
vs.Unauthorized
errors, it's confusing enough as it is already, I don't take any shortcuts.
Unauthenticated Error Flow
The errs.Unauthenticated
error should only be raised at points of authentication as part of a middleware handler. I will get into application flow in detail later, but authentication for go-api-basic
happens in middleware handlers prior to calling the app handler for the given route.
- The
WWW-Authenticate
realm is set to the request context using thedefaultRealmHandler
middleware in the app package prior to attempting authentication. - Next, the Oauth2 access token is retrieved from the
Authorization
http header using theaccessTokenHandler
middleware. There are several access token validations in this middleware, if any are not successful, theerrs.Unauthenticated
error is returned using the realm set to the request context. - Finally, if the access token is successfully retrieved, it is then converted to a
User
via theGoogleAccessTokenConverter.Convert
method in thegateway/authgateway
package. This method sends an outbound request to Google using their API; if any errors are returned, anerrs.Unauthenticated
error is returned.
In general, I do not like to use
context.Context
, however, it is used in go-api-basic to pass values between middlewares. TheWWW-Authenticate
realm, the Oauth2 access token and the calling user after authentication, all of which arerequest-scoped
values, are all set to the requestcontext.Context
.
Unauthenticated Error Response
Per requirements, go-api-basic does not return a response body when returning an Unauthenticated error. The error response from cURL looks like the following:
Unauthorized Errors
The errs.NewUnauthorizedError
function initializes an UnauthorizedError
.
Unauthorized Error Flow
The errs.Unauthorized
error is raised when there is a permission issue for a user when attempting to access a resource. Currently, go-api-basic
’s placeholder authorization implementation DefaultAuthorizer.Authorize
in the domain/auth package performs rudimentary checks that a user has access to a resource. If the user does not have access, the errs.Unauthorized
error is returned.
Per requirements, go-api-basic does not return a response body when returning an Unauthorized error. The error response from cURL looks like the following:
Originally published at https://dangillis.dev on June 21, 2021.