Ok, this is the last one. Setting up the programs to run by anyone, not just me.

The main thing about the weather API saga is that I compile the Go code on my computer, download the data to my computer, run queries on my computer, etc. It's all on my computer, then I deploy the appropriate files like the new cities.json and countries.json to production.

It's not ideal.

If you haven't yet read, I'm talking about my Weather API series, you can read here:

Part 1
Part 2
Part 3

So, I set up a folder on production with the files needed to build out the cities and countries files. And wrote a script to execute the appropriate steps in the appropriate order.

But of course, I had to make it easier for someone new coming in. Step 1 is that there has to be a csv of unique locations from the database that the geocodecsv program reads. Rather than having someone have to provide that, I wrote a simple utility that can take a query and a connection string, and output the data as a csv. I called it sqlcsv!

This executes the query in part 3 of the series, "select distinct city, state, country from tournaments", and just writes it to the unique_locations.csv file.

The most modifications to the code were in geocodecsv, where I am now writing a json file of any cities that it couldn't geocode. This will be helpful later on. I also am writing the countries.json file from that code, for some reason I wasn't doing that before, but will definitely make it easier. I was, however, writing the countries.json from the geonames program.

So my script is this:

1. Execute sqlcsv to generate a csv file with the distinct locations.
2. Execute geocodecsv to generate a new cities.json and countries.json, as well as write out the same format json for any cities that weren't able to be located.
3. Loop through any cities that weren't geocoded and just output them. (Let them deal with it :P )

There are instructions in a readme file to add cities to the appropriate file (add or modify) depending on the circumstances. I leave that pain in the butt to people in the future that stumble across this script.

If you're curious, reading a json file in powershell looks like this. It uses dynamic objects, so you don't have to define the properties that each record can have.

$notfound = Get-Content -Raw -Path notfoundcities.json | ConvertFrom-Json
Write-Host "the following locations were not able to be geocoded"
Foreach ($loc in $notfound) {
    Write-Host $loc.Name $loc.State $loc.Country
}
 

More Thoughts

In part 3, I mentioned that I wanted to make the geocodecsv a separate program instead of modifying the geonames program. This is still the right decision. There is a case, though, that you could opt to have a UNIX style set of programs where one output is another's input. This is practically the case in this instance. I could have written a "outputJson" Go program that reads in the JSON file and outputs a defined set of properties or whatever. However, powershell is capable of this.

However, I'd really like to be able to do true UNIX style programming, but it's Windows. Granted, you can grab the output of a program into a variable, conceptually 

$unique_locs_csv = (execute sqlcsv)

Then pass that in to geocodecsv  like

geocodecsv [...] < $unique_locs_csv

There are probably ways to do this in Windows.

There's another problem with the files, because the file exists from the last time I ran it if there was an error that prevented it from running it this time. But, that could be solved with just deleting the files at the end of the script...

That's all. Happy coding!