Lessons through failure. Episode 1
I fucked up this last week.
On Monday, our primary contact for a large client sent over some last minute requirements and deadlines that were needed by end-of-day Wednesday. I didn’t have a lot of time to collect requirements and execute it without having to rearrange my priorities. But, I accepted the challenge.
The big change involved was that we were going to be supplied with a ton of data to be imported in to the database and approximately 20% of the data provided was new records, while the rest were duplicates. However, the other 80% wasn’t to be discarded as there were a few attributes that needed to be updated from the data file (which was supplied from the client’s parent company). In my haste to get the task done on time (didn’t get proper export file to be imported in our system until Wednesday morning)... I ended up running a few tests locally and pushed it out to production.
I managed to get the import file to run in production before leaving on Wednesday afternoon. The following morning, I came into the office to find out that my import process didn’t match up records properly and resulted in nearly all of the 80% side of that to be duplicated in the system. This resulted in lost productivity for our client, their vendors, and our team over a 12 hour period as people were confused about why reports were running weird, online transactions didn’t account for the duplicated, etc.
It took me most of Thursday and Friday to clean up the data that got skewed due to that oversight. Hi ho.
So, the take away from this? Sure, I could have blamed it on a lack of sufficient time to properly test things, but that’s bullshit. I should have had at least one other developer from our team review the problem and evaluate my proposed solution prior to me attempting to push into production.
Luckily, the client was happy that we were able to finish the last minute tasks, despite the unexpected headaches that cropped up.
If anything, I was just disappointed in myself, but Alex reminded me how important it was to fail early, fail often. It didn’t kill me (or anybody else for that matter), cost us the project, nor was it irreparable.
In the real world, deadlines and requirements change on a moments notice and it’s experiences like this that will make ourselves more confident that we can quickly respond to and execute.
What was your latest failure?
