Today I would like to briefly introduce my co2monitor.api project. This is a REST API that allows me to store and read the readings from my CO2 monitor.
The CO2 monitor
My CO2 monitor is the TFA Dostmann CO2 monitor AIRCO2NTROL MINI 31.5006. This measures not only the CO2 concentration but also the temperature. The measured values are shown alternately on a small display, the CO2 value is still shown next to it on a traffic light, with green up to 799ppm, orange up to 1199ppm and red from 1200ppm. How to read this out has been described in detail by Henryk Plötz in All your base are belong to us. My CO2 monitor is connected via USB to a Raspberry Pi 3B, on which I run the following readout script.
The readout script
There are numerous implementations of the readout script, I chose CO2Meter from heinemml at the time. My readout script embeds the CO2Meter script, reads the data every minute and then sends it to the REST API. Clean code and best practices are not in this script, as I wrote it back when I was a beginner and haven’t touched it since. I just tweaked it briefly to make it work with the new co2monitor.api.
The old REST API
I had implemented the old REST API in Python using FastAPI. It stored the readings in a SQLite database and main could read them via a GET route. I have now replaced the old API with the new co2monitor.api.
Once everything new
Through education, I have come to appreciate statically typed programming languages. However, However, I have enough C# at work and I wanted something new, which is similarly easy to use as Python, statically typed and very fast in the development process. That’s why I decided to go with Go. Go is a statically typed programming language that stands out for its simple syntax and performance. After a crash course with Learn Go with Tests, I got to work.
The basic framework:
- Gin Web Framework: A very simple and fast web framework for Go
- GORM: An ORM for Go, I was spoiled by Entity Framework in C#
- Compile Daemon for Go: A daemon that automatically recompiles and restarts the server after each save
- Log: A logger that offers excellent configuration options
- PostgreSQL: A relational database that is most appealing to me
- Docker: To run the application in a container
The highlights of the implementation
I don’t want to bore anyone here, so I’ll just briefly discuss the highlights of the implementation. The project can be found on GitHub, if you are interested in the details, feel free to check out the code.
The database connection, creating the schema
Using GORM, I connect to a PostgreSQL database. I assumed that the database with all tables will be created automatically by GORM. For this I had used the connection template from the GORM documentation.
func ConnectToDb() {
dsn := os.Getenv("DATABASE_URL")
db, err := gorm.Open(postgres.New(postgres.Config{
DSN: dsn,
PreferSimpleProtocol: false,
}), &gorm.Config{
Logger: logger.Default.LogMode(logger.Info),
NamingStrategy: schema.NamingStrategy{
TablePrefix: "co2monitor.",
SingularTable: false,
},
})
if err != nil {
log.Fatal("Failed to connect to database. \n", err)
}
// creating the schema
log.Info("Connected to database.")
DB = DbInstance{
Db: db,
}
}
By specifying the schema, I thought that it would be created as well. But this was not the case, not even by db.AutoMigrate(&models). I then adjusted the function and now create the schema manually if it does not exist.
dbSchema := os.Getenv("POSTGRES_DB")
createSchemaCommand := fmt.Sprintf("CREATE SCHEMA IF NOT EXISTS %s;", dbSchema)
result := db.Exec(createSchemaCommand)
if result.Error != nil {
log.Fatal("Failed to create schema. \n", result.Error)
}
The routing and the middleware
Since I don’t want everyone to be able to use the API, I included middleware that checks the request. I wanted a permission concept that distinguishes GET from POST/PATCH and DELETE routes and only allows access with different keys on the respective routes. The middleware checks the API key supplied in the header and returns a 401 Unauthorized if the key is invalid or if a route is accessed without sufficient access rights. I first tried nesting the middlewares. I thought I could write a middleware as a higher-order function to pass the required API key.
func locationRoutes(superRoute *gin.RouterGroup) {
controllers := &controllers.APIEnv{
DB: db.GetDB(),
}
locationRouter := superRoute.Group("/location")
adminMiddleware := locationRouter.Group("/")
adminMiddleware.Use(middleware.RequireAuth("X_API_KEY_ADMIN"))
{
adminMiddleware.POST("/new", controllers.CreateLocation)
getMiddleware.GET("/", controllers.GetLocations)
// other methods
}
normalMiddleware := locationRouter.Group("/")
normalMiddleware.Use(middleware.RequireAuth("X_API_KEY"))
{
normalMiddleware.GET("/", controllers.GetLocations)
// other methods
}
}
But that didn’t work for my case, or rather I would then have had to repeat myself for the GET methods, since someone with the admin key should also be able to query the GET routes. Therefore, I implemented the middleware to check the request and API key with the appropriate permission or return a 401.
func RequireApiKey(c *gin.Context) {
APIKey := c.Request.Header.Get("X-API-KEY")
adminAPIKey := os.Getenv("X_API_KEY_ADMIN")
normalAPIKey := os.Getenv("X_API_KEY")
if APIKey == "" {
c.AbortWithStatusJSON(http.StatusUnauthorized, gin.H{"message": "Unauthorized"})
log.Infof(`API-Key was not found or wrong. API-Key: <%s>; URL: <%s>`, APIKey, c.Request.URL)
return
}
switch c.Request.Method {
case http.MethodGet:
if APIKey == adminAPIKey || APIKey == normalAPIKey {
c.Next()
return
}
case http.MethodPost, http.MethodPatch, http.MethodDelete:
if APIKey == adminAPIKey {
c.Next()
return
}
}
log.Infof(`Unauthorized API-Key. API-Key: <%s>; URL: <%s>; Method: <%s>; Path: <%s>`, APIKey, c.Request.URL, c.Request.Method, c.FullPath())
c.AbortWithStatus(http.StatusUnauthorized)
}
This then also led me to a much cleaner routing. 🥳
func locationRoutes(superRoute *gin.RouterGroup) {
controllers := &controllers.APIEnv{
DB: db.GetDB(),
}
locationRouter := superRoute.Group("/location")
locationRouter.Use(middleware.RequireApiKey)
{
locationRouter.GET("/", controllers.GetLocations)
locationRouter.GET("/search", controllers.GetLocationBySearch)
locationRouter.POST("/new", controllers.CreateLocation)
locationRouter.PATCH("/:id", controllers.UpdateLocation)
locationRouter.DELETE("/:id", controllers.DeleteLocation)
}
}
The unit tests
In my training, the topic of unit tests unfortunately came too short, but I consider tests essential for good, maintainable code. At my current employer we test intensively, so it was also my ambition to cover the application with unit tests. The included testing module in Go is very good in my opinion, with Testify it becomes superior as assertions can be written as one-liners. The functions that gave me the most headaches were those that use database binding. In .Net, I know the SQLServer’s InMemory database makes testing database operations really easy. The initial search along the lines of “How do you test Go functions that use gorm?” didn’t have the best hits, unfortunately. Some exotic suggestions (🤔) like TestContainer or testing against a real database had me briefly worried I shouldn’t have used GORM after all. But I didn’t have to search too hard, I just had to read the GORM documentation for connecting a database, there was a reference to the Pure Go SQLite driver for GORM. SQLite provides the possibility of an InMemory database, which allowed me to write my database mock.
type BaseFixture struct {
Db *gorm.DB
}
func (f *BaseFixture) Setup(t *testing.T) {
var err error
f.Db, err = gorm.Open(sqlite.Open(":memory:?_pragma=foreign_keys(1)"), &gorm.Config{})
require.NoError(t, err)
f.Db.AutoMigrate(&models.Location{}, &models.Co2Data{})
}
func (f *BaseFixture) Teardown(t *testing.T) {
f.Db.Session(&gorm.Session{AllowGlobalUpdate: true}).Delete(&models.Location{}, &models.Co2Data{})
}
func (f *BaseFixture) AddDummyData(t *testing.T) {
f.Db.Create(&Locations)
f.Db.Create(&CO2)
}
The SQLite database is recreated for each test and deleted after the test. I also wrote a function that writes dummy data to the database so that I can use it in the tests. A test then looks like this, for example:
func TestGetLocationBySearch_ShouldFindOneById(t *testing.T) {
f := tests.BaseFixture{}
f.Setup(t)
f.AddDummyData(t)
defer f.Teardown(t)
result, err := db_calls.GetLocationBySearch(f.Db, "2", "")
require.NoError(t, err)
assert.Equal(t, tests.Locations[1].Name, result[0].Name)
assert.Equal(t, 1, len(result))
}
Unit tests ✅
Swagger
With the ease of including Swagger in .Net, I’m used to having documentation for my API with one click. There is also an easy way to use Swagger for gin-gonic by including gin-swagger. If you then write the documentation of the routes accordingly as a comment above the functions you have the documentation with one click. 🧐
Deployment and hosting
As mentioned in my previous blog post Hosting with Oracle Cloud Free Tier, the API runs in a Docker container on an Oracle Cloud Free Tier compute instance. The API is accessible and documented via co2.leyrer.io.
Conclusion
I wrote a REST API in Go that I covered with unit tests and can run in a Docker container. I learned a lot and am very happy with the result. 🥳