Overwhelmed in the Presence of Uncertainty May 14, 2023
When faced with uncertainty, I tend to work long hours
Software estimates are hard. This isn't a secret or new information. This process is also not unique to me. But when asked how long I think something will take, there are many variables that go into the equation. I'll try to speak in concrete examples.
On a project with work, the client asked for a user system, logging in, editing profiles, saving favorite 'things'. This is a migration into the current .NET based system from their existing Drupal based system. It's not important what these are so you can Google them on your own. But they are very different. The decision made before I was put on the project was to use Amazon's Cognito. I messed around with it on a previous project, but it did not end up being used so I don't have real knowledge and "production" knowledge of it. Production knowledge is when you wrote something, deployed it for thousands of people to use, and learned first hand as to the many hundreds of ways it can break :P
In light of this uncertainty, I freakin worked. We have allocations at work, and I'm typically allocated about 25% of the week to my big client, and maybe 25% of the week to another, and this project was the rest (I was technically only allocated 30 hours of 40, so I determine which project needs the extra love and put it towards that). We also do timesheets, and I don't really ever put in more than 40 hours, even though I might work 50-60. Well, the week of Cognito was definitely one of those times. I got it working, got 'favorites' saving, loggin in and out. There's no signup requirement (odd), I think it's handled manually.
I think the part that's missing is I'm not a front end developer, I don't dabble in the newest front end tech often, though I do have extensive history with it, so I can work my way around it. Cognito can be backend, but I think it's easier to just use Amplify and separate it out from the backend. It'll be like users don't even exist! But part of getting it all up was reading through hundreds of pages of documentation. Although after 20 years in this field, I can look at a page in a second and know if it has what I need. I had to install node.js (ugh), Amplify, webpack, the aws command line so I can reset my test user's password since I didn't have front end built to handle that. It was a lot of work, but I was determined to end the week having a good idea what it was going to take, and having a working demo. I succeeded.
The next bit is with the whole project in general. The backend part. This one is a bit trickier since it involves humans. I use the "20 years in the field" a bit because it's new. I think I started working in 2002 actually. Anyway, after 20 years, I've found there are efficient and correct ways to gather and document requirements, and as a self proclaimed "master logician" :P (I sorta kid), there are things that make sense and really ties things together, and there is the wrong way. This stems from something very specific. I don't blame the process of gathering requirements, I don't blame the people involved in that process, I blame it on the fact that the requirements are only front-end based. I'll try to explain.
The current project has a "resource" requirement. This is not defined explicitly except through designs. What I gathered is that it has a title, body text, image or video(s), associated downloads, tags, and a date. The project also has a resource search page which is also currently defined through designs. It has a tag filter, and I gathered from this design that tags can be organized into categories. Just looking at these two things, I felt like I had about 100 hours of work ahead of me that really hasn't been accounted for in the estimate. Faced with these uncertainties, I freakin worked.
Over the past Friday and Saturday (!!), I put in 20-24 hours. I built the resource type, worked in a tagging system, built search. Sitecore has the concept of a wildcard item, and I researched and implemented the url builder for resources, and the item resolver. These now work. A lot of the functional requirements were assumptions. I just did it the way I thought it needed to be done. I had to, in order to not feel overwhelmed. Now I feel good.
This is where the overwhelmed feeling comes from. A lot of variables, things only defined by what data they have and how they will look, and a lot of unknowns. I need all of the information. How will it connect to the rest of the site? What is this thing? I need to know. Leave me with as little unknowns as possible and I will craft a solution for what is known, and probably leave it a little bit flexible for what is not. And when I feel overwhelmed, my estimate can be like "100 hours!" I think on these two things I spent about 40-50. Still a ton, and a lot more work on the project. I have a lot of hours left on the project and it lasts until the first week of July. And now I feel I'm through a decent chunk of it.
This is also a problem with AGILE which I hate and don't find useful at all. With agile, you have to break the project down into week sized chunks. But "figure out and develop a user system based on Amazon's Cognito" and "we need a whole resource system built with searching and permissions" don't really fall into that, and a lot of other things are based on those things being there. But the other things fell onto the first few weeks of development. I felt those two had to be done first and I did that.
Some things are still a mess. It's an old project, it uses an old front end system we had designed. The new webpack based JS with Amplify doesn't go with it, so we'll have two different systems running. I rewrote a lot of the backend and cleaned up the data. I see a messy project and I do my most work. This can make up for all the times I day trade while working on my "properly implemented" projects that need no changes :)
Now I can enjoy the rest of my weekend, go spend time with Mom, watch the Sixers in Game 7. And I'll be back at it tomorrow, but I'm in a much better place.