And on my general overview of test driven development

Testing software is very important. Although I've been known to see a bug, modify the code, and then commit it to let the CI/CD push it out and it's fixed. Notice I skipped the parts of 1. making sure it compiles, and 2. testing it. Sometimes it's just obvious.

Test Driven Development, or TDD, is a paradigm where you write tests for every method in your application, and you often run these tests to make sure you haven't broken anything. There's a concept of test coverage which measures how much of your application is covered by tests. There are many tools and frameworks that will help build your tests, and some automatically run your tests.

One type of TDD is "unit testing". This is testing individual functions which don't rely on any external sources other that what can be passed in from the test. I wrote some unit tests for a method that ran the MOD10 algorithm to verify if a credit card number was valid. This is a good case for a unit test.

Another type of TDD is "integration testing". This is where your tests need to connect to a database or some API to get the data. I don't see a need to use a TDD framework to write these kinds of tests. What you should do is write your own thing that makes the software actually work against the data source. This is typically what test environments are for, and make sure to supply various forms of data so that as many cases that you can think of are covered.

I don't really subscribe to TDD. As such, my description above is probably way off. I will write some tests, however. In go it's very easy to run "go test -v" and all of your tests run. This is good for some things where I'm doing calculations or parsing something. Typically I don't see a use in my software for TDD. It is mostly reading from a database and rendering that data in some way. This is all integration testing.

At work, we're doing a huge project. The main problem is it's all integration. We have a few data sources which end up being databases, and the backend provides an API to access all of the data. These are all REST-like services. Mainly differing in the part where I require all POST requests, so that we're not using the query string. And while a body can be attached to a GET request, it shouldn't mean anything. And that would just be weird, honestly.

As of this writing, there are 80 or so Route() attributes on various controller methods, but some are not used by front end. In the end there might be 100 or so? Guessing.

The general structure is something like, one endpoint returns a list of things, and then there are a few endpoints that can be called with some data from the list request as their input. I think that's probably a widely used structure. Microservice architecture or just plain old REST-like things. Front end cannot write any data, they are all read only.

Some of these are nested even further down. So with a property, you can call another service and get a new list of properties, which can be used in a call to another service. Again, this is probably a widely used structure, with hierarchy. Examples fail to come to mind because I've been in my golf world for the past year.

So I will share a secret that I don't really share on this site. I'm the lead backend developer for a professional sports league's website. It is to do with golf. So my testing is all golf related. The main data that I'm interested in testing have to do around tournaments and players. Tournaments can have start and end dates, past results, past winners, and some general website things like headers and images, tables, etc.

To test a tournament, I would call the API which gets a list of tournaments. Each of those tournament results have a tournament code and a tournament id. The code doesn't change throughout the years, and tournament id is that year's instance. The endpoints that I want to test are to get the page, the header, an "info line", the weather, TV times, past winners, and then the years that that tournament has been played. With the years, I want to test the results endpoint for each year.

So I wrote "trest". It is of course a conglomeration of "test" and "rest". It uses a custom file format. Hierarchical, as required via my description of hierarchical REST endpoints. This is the format:

Test main
 Method POST
 Headers
  Var Content-Type application/json string
 Variables
  Var Year 2024 int

 Test tournaments result 
  Url /-/tournaments/list {}
  Var id tournamentId int
  Var code tournamentCode string
  Var link link.href string
  Test page result
   Url /-/page/json?url=$link {}
  Test header result
   Url /-/tournaments/header { "code": "$code" }
  Test info line result
   Url /-/tournaments/infoline { "code": "$code" }
  Test weather result
   Url /-/tournaments/weather { "tournamentId": $id }
  Test broadcasts result
   Url /-/broadcasts/schedule { "tournamentCode": "$code" }
  Test pastwinners result
   Url /-/statistics/tournamentpastwinners { "tournamentCode": "$code", "count": 100 }
  Test years result
   Url /-/tournaments/years { "code": "$code" }
   Var y @index int
   Test results result
    Url /-/tournaments/results { "code": "$code", "year": $y }

The source for trest can be found on github. I will add a readme at some point :)

Basically, it reads in the file, for each top level Test element, it runs the URL with the Body. Variables are grabbed from the response and passed down so they are in scope for all children tests.

Each test provides the parent property, like "Test years result". Mostly an array of object is handled, but in the case of an int array, like in /-/tournaments/years, it's specified that we want each element to be stored, which is just specified with @index.

This is very specialized to my needs currently. It can dig into objects within each object, like with link.href to get the href from the link object of each tournament. Outside cases aren't tested :)

I had to bump up my IIS max queue length to 20000 to be able to handle the amount of concurrent requests I pass to it :D Ah, go concurrency is too easy!

Happy coding!