REST API Error Handling in Go
Handling errors is really important in Go. Errors are first class citizens and there are many different approaches for handling them. Initially I started off basing my error handling almost entirely on a blog post from Rob Pike and created a carve-out from his code to meet my needs. It served me well for a long time, but found over time I wanted a way to easily get a stacktrace of the error, which led me to Dave Cheney’s https://github.com/pkg/errors package. I now use a combination of the two. The implementation below is sourced from my go-api-basic repo, indeed, this post will be folded into its README as well.
My requirements for REST API error handling are the following:
- Requests for users who are not properly authenticated should return a
401 Unauthorizederror with a
WWW-Authenticateresponse header and an empty response body.
- Requests for users who are authenticated, but do not have permission to access the resource, should return a
403 Forbiddenerror with an empty response body.
- All requests which are due to a client error (invalid data, malformed JSON, etc.) should return a
400 Bad Requestand a response body which looks similar to the following:
- All requests which incur errors as a result of an internal server or database error should return a
500 Internal Server Errorand not leak any information about the database or internal systems to the client. These errors should return a response body which looks like the following:
All errors should return a
Request-Id response header with a unique request id that can be used for debugging to find the corresponding error in logs.
All errors should be raised using custom errors from the domain/errs package. The three custom errors correspond directly to the requirements above.
Typically, errors raised throughout go-api-basic are the custom
errs.Error, which looks like:
These errors are raised using the
E function from the domain/errs package.
errs.E is taken from Rob Pike's upspin errors package (but has been changed based on my requirements). The
errs.E function call is variadic and can take several different types to form the custom
Here is a simple example of creating an
When a string is sent, an error will be created using the
errors.New function from
github.com/pkg/errors and added to the
Err element of the struct, which allows retrieval of the error stacktrace later on. In the above example,
Code would all remain unset.
You can set any of these custom
errs.Error fields that you like, for example:
Above, we used
errs.Validation to set the
errs.Code represents a short code to respond to the client with for error handling based on codes (if you choose to do this) and is any string you want to pass.
errs.Parameter represents the parameter that is being validated or has problems, etc.
Note in the above example, instead of passing a string and creating a new error inside the
errs.Efunction, I am directly passing the error returned by the
errs.E. The error is then added to the
github.com/pkg/errorspackage. This will enable stacktrace retrieval later as well.
There are a few helpers in the
errs package as well, namely the
errs.MissingField function which can be used when validating missing input on a field. This idea comes from this Mat Ryer post and is pretty handy.
Here is an example in practice:
The error message for the above would read title is required
There is also
errs.InputUnwanted which is meant to be used when a field is populated with a value when it is not supposed to be.
Typical Error Flow
As errors created with
errs.E move up the call stack, they can just be returned, like the following:
In the above example, the error is created in the
outerreturn the error as is typical in Go.
You can add additional context fields (
errs.Kind) as the error moves up the stack, however, I try to add as much context as possible at the point of error origin and only do this in rare cases.
At the top of the program flow for each service is the app service handler (for example, Server.handleMovieCreate). In this handler, any error returned from any function or method is sent through the
errs.HTTPErrorResponse function along with the
http.ResponseWriter and a
errs.HTTPErrorResponse takes the custom error (
errs.UnauthorizedError), writes the response to the given
http.ResponseWriter and logs the error using the given
returnmust be called immediately after
errs.HTTPErrorResponseto return the error to the client.
Typical Error Response
errs.HTTPErrorResponse writes the HTTP response body as JSON using the
When the error is returned to the client, the response body JSON looks like the following:
In addition, the error is logged. If
zerolog.ErrorStackMarshaler is set to log error stacks (more about this in a later post), the logger will log the full error stack, which can be super helpful when trying to identify issues.
The error log will look like the following ( I cut off parts of the stack for brevity):
Ewill usually be at the top of the stack as it is where the
errors.WithStackfunctions are being called.
Internal or Database Error Response
There is logic within
errs.HTTPErrorResponse to return a different response body if the
Database. As per the requirements, we should not leak the error message or any internal stack, etc. when an internal or database error occurs. If an error comes through and is an
errs.Error with either of these error
Kind or is unknown error type in any way, the response will look like the following:
The spec for
401 Unauthorized calls for a
WWW-Authenticate response header along with a
realm. The realm should be set when creating an Unauthenticated error. The
errs.NewUnauthenticatedError function initializes an
I generally like to follow the Go idiom for brevity in all things as much as possible, but for
Unauthorizederrors, it's confusing enough as it is already, I don't take any shortcuts.
Unauthenticated Error Flow
errs.Unauthenticated error should only be raised at points of authentication as part of a middleware handler. I will get into application flow in detail later, but authentication for
go-api-basic happens in middleware handlers prior to calling the app handler for the given route.
WWW-Authenticaterealm is set to the request context using the
defaultRealmHandlermiddleware in the app package prior to attempting authentication.
- Next, the Oauth2 access token is retrieved from the
Authorizationhttp header using the
accessTokenHandlermiddleware. There are several access token validations in this middleware, if any are not successful, the
errs.Unauthenticatederror is returned using the realm set to the request context.
- Finally, if the access token is successfully retrieved, it is then converted to a
GoogleAccessTokenConverter.Convertmethod in the
gateway/authgatewaypackage. This method sends an outbound request to Google using their API; if any errors are returned, an
errs.Unauthenticatederror is returned.
In general, I do not like to use
context.Context, however, it is used in go-api-basic to pass values between middlewares. The
WWW-Authenticaterealm, the Oauth2 access token and the calling user after authentication, all of which are
request-scopedvalues, are all set to the request
Unauthenticated Error Response
Per requirements, go-api-basic does not return a response body when returning an Unauthenticated error. The error response from cURL looks like the following:
errs.NewUnauthorizedError function initializes an
Unauthorized Error Flow
errs.Unauthorized error is raised when there is a permission issue for a user when attempting to access a resource. Currently,
go-api-basic’s placeholder authorization implementation
DefaultAuthorizer.Authorize in the domain/auth package performs rudimentary checks that a user has access to a resource. If the user does not have access, the
errs.Unauthorized error is returned.
Per requirements, go-api-basic does not return a response body when returning an Unauthorized error. The error response from cURL looks like the following: