Eight years ago when I started with this company, there where 2 servers. A Linux box with a db and php with a bunch of cron jobs and some ancient code from an old application with a website bolted on the side of it. The Windows box had some aspx files that did some ridiculous garbage with an off the shelf ftp application that downloaded logs that were parsed and stuck in a horribly designed mssql db. There was not a single piece of code that was backed up anywhere, not even for the mobile applications.
Since that time. I have implemented using git on all new projects and we currently have 19 codebases in separate repositories. The windows server has been replaced and we run on an all linux platform. We have moved to a set of virtual machines that share a backend network which is how they communicate with the dedicated db server. In the meantime, I have designed and architected a process to create automated voiceovers for videos. As an AI project I designed, architected, trained and wrote a program to recognize vehicle parts in a video and a way to jump to that place in the video when a button is pressed.
We currently have 12 servers that I take care of in two separate data centers. We have multiple API endpoints that I have created and maintain for custom connections with outside companies and with our internal application. I have enforced better coding practices. Updated outdated applications and code added more security and sanity to everything that exists for this company.
I researched and introduced a stable and state of the industry framework (Laravel) to base any new portal work on. It has allowed us to standardize and add some much needed layers of security and code order to some very backwards and poorly written old code bases. New products and clearer communication has been enabled with our new partner portal.
This is all just for the one company. I also have been supporting a second venture for the same owner. 6 more servers built from scratch, another CDN solution, two more mobile applications and an added layer of technology (Elastic Search). Every API endpoint was written by me and most of the database was build by me. All this code exists in repositories and we also had to integrate with an AR program with multiple models. So far I have not received a single PENNY for any of that work for the past 4 years.
I have given this company my all and unquestionably gone above and beyond any and every request. I almost make the same salary I made as a Development Manager for a hosting company 15 years ago. And even with that the benefits I had back then are 10 times better that I have now. Sometimes I wonder why I am not happier than I am. I am grateful for my life, but I EARN IT every day.
So this morning when I make the decision to upgrade a severely overloaded box before it crashes and takes down the database again, I don’t expect to be berated online by the boss. NOTHING bothers me more than someone second guessing me when I am doing my job. Being the person who is responsible for systems that I also designed, and built myself. I don’t expect someone else to argue with me when I make a decision about them. I know full well that people need to use them during business hours. I also know that when it is using 97.6% of the memory and literally 100% of the processor, it is about to die. Unfortunately this box has a tendency to take down the db with too many connections when it is panicking.
So after the online verbal abuse I simply let him know he was wrong. That of course will trigger him since he can never be “wrong”. So after a discussion I agree to wait to upgrade the box. Well I didn’t have to wait long. A few hours later the database starting having issues. When this happens, it causes our applications to start failing. This can literally negatively effect HUNDREDS of websites and a significant number of people who are actively using our applications.
With the outage happening. I went ahead and updated the box to one double its previous capacity, but it was too late for the db and one of the app servers. Other changes had been made (not by me) while guessing how to fix the issue. All in all we ended up being down for about 30 minutes or possibly an hour impact that should never have happened. It will never be admitted that this is the case but I don’t care at this point because the server has been upgraded.
It is just passed midnight on a Friday night and as I write this i am also logged into the db server running some update scripts. This job will never end, until I walk away.
