History

The most money I have ever made was my years spent selling motorcycles. Software Engineering is by all means a lucrative profession, and I’ve never been the starving artist developer who prizes a compelling mission over a compelling paycheck. But the years I sold motorcycles - between 2003 and 2008 - coincided with a wave of temporary insanity in the US real estate market. Heretofore normal people became mad with the sudden access to “wealth,” in the form of massive home equity reserves they could spend as easily as cash via HELOC checkbooks. And spend they did - in my dealership, to the tune of $100,000 boats, $50,000 Harley Davidsons, and $30,000 jetskis, often two or three at a time (“get the pair or the garage will look lopsided”). This was a time when ex-convict, high-school dropout motorcycle salesman would net twice the income of the Ivy-league lawyer to whom they were selling an overpriced jetboat. As salespeople we assumed this prosperity to be an eternal constant, and we planned, spent, and generally behaved accordingly. The day after Lehman Brothers toppled in the fall of 2008, I checked that the front doors of the dealership were unlocked. Twice. Before that day, I could look out the window of my elevated office and see thousands of customers spread across our dealership floor, a football field of motorcycle commerce in action. From that day forward, the customers numbered in single digits and the floor was populated only by listless, terrified salespeople. Salespeople that saw their incomes vanish and their savings evaporate; most went from six-figure incomes to earning little more than minimum wage overnight. But a short time into this barren economic winter, there was a sudden reprieve in the form of the Southern California gas price explosion. Economic forces combined to send the price of a gallon to just under $5 (that’s over $7 a gallon today), and Angelinos again panicked - this time stampeding to our dealership to purchase 70-plus-mpg scooters and motorcycles. At one point we partnered with a local car dealer so customers could trade in their massive SUVs and lifted trucks (for a sizable loss) and pay top dollar for a Vespa clone. By all accounts the gravy days were back, and most of the salespeople resumed decadence as usual. But the seasoned ones, the wise old oaks (probably in their 30’s now that I think of it) maintained caution. They used this temporary reprieve to right-size, to reposition themselves. One sold his mansion (there is no other word for that house) to an eager young co-worker for a sizable, but still tolerable, loss. Many left management roles and returned to the showroom floor, preparing for the impending downsizing (and reduced need for management overhead). Of course the scooter bubble quickly burst, and what was left was nearly half a decade of the worst sales and slowest rebound in the history of power sports. It was only the seasoned veterans that survived.

History Repeats Itself

I feel I will tell a very similar story of Software/Data/DevOps (et al) Engineering in the not-to-distant future. A slow-burning period of unsustainable gluttony followed immediately by a devastating correction, then a moment of opportunity, mistaken by most to be a return of “the good times”, and finally, a long plunge into sensible reality. The first two parts, past decadence and inevitable correction, are common narratives at the moment so I’ll only touch on them enough to set the stage. What I think matters today, and the heart of this writing, is a dissection of that critical third part - the window of opportunity - and the sanity that is to follow.

It was the best of times

We all know the stories of zero-rate excess. (M/F)AANG employees outraged that their favorite cereal is no longer stocked at the closest snack station, CompSci grads dumbfounded that their first job would expect a five-day work week. Then there are the more mundane (but no less tell-tale) evidences: recruiters hounding anyone titled “Developer” with sight-unseen six-figure job offers, lavish office perks that sounded more like a cruise ship than a place of business (“we have a climbing wall!”). Rock-star impunity as “the talent,” in great demand and impossible to replace. And of course, a shared cognitive dissidence ignoring the lack of concrete ROI from all of it.

It was the worst of times

We all know these stories too. (M/F)AANG tech layoffs in the range of 40,000 people with more on the way. 320,000 across all of tech in the last year. VC funding at the lowest point in six years, 62% lower than the funding peak in ‘22. Layers of software management stripped away, initiatives shit-canned, and post after painful social media post from displaced big tech Engineers who can’t seem to find their next role. All the while, a looming fear of Artificial Intelligence that might quickly automate away roles that we once believed to be sacredly human.

The Indian summer

During these dark times OpenAI grew from $29B to over $80B in value. Twenty new AI unicorns were minted in ‘23 (the slowest year for new unicorns in half a decade). The party has not ended for Engineers wishing to join the AI cash grab; simply wrap OpenAI’s API with a few lines of JavaScript and sprint to your nearest investor with an open wallet. “Look my app summarizes {thing}!” “My chat app has a personality like John Wick!” and the checks pile up.

But remember I called this a moment of opportunity. Not for a short-lived buck, but an opportunity to carve out both soft landing and safe passage into the future. An opportunity not available to everyone; in fact, what makes the largely untenable nature for so many Engineers make it an opportunity for others.

AI is a big space, but let’s narrow that view to predictive and generative models for the sake of conversation. If you look past the generative AI hype and ask what the larger set of AI tools offer, the answer is pretty simple - these tools offer a way to make computers do things that are really hard (or functionally impossible) to do without them. For a seasoned programmer this is not a new paradigm; every time you have adopted a new tool or learned a new language, it was because doing so expanded your plane of influence. Consider: at some early point in your career, you probably wanted to keep state between runs of your program but didn’t know how. So you learned how to interact with a database. Instantly, something that was impossible before (saving state) was now quite reasonable. It wasn’t free knowledge or magic - adding state to a program (along with the complexity of database adapters and idiosyncrasies) introduces new challenges, new elements for you to master in your design. But this ability of your program was unlocked. Today, you might want the results from your program to improve based on feedback from users. So you learn how to create an ML training cycle, applying user-provided scores to a dataset which is fed back to re-train your model. Instantly, something that was impossible before (progressive self-improvement of your program’s output) is now quite reasonable.

Database and ORM evolution demonstrates a useful parallel; in the late 1980’s database innovation came from commercial powerhouses like Microsoft SQL Server and Oracle, and from OSS projects like Postgresql (and later MySQL). With the focus on OOP during the dot-com era, the first ORM and MVC patterns (both in Smalltalk) arrived in 1995. Even so, in 2004 many of us were still building PHP PDO models that amounted to 10,000 lines of stringified SQL queries. When you use any ORM today, you expect to be able to do things like

user.save()
user.reset_password()

intuitively, and not hand-craft queries like

database.run_query("UPDATE user SET(u.first_name,
									u.last_name # etc for 30 more lines`

The proliferation of MVC + ORM frameworks like Spring, Ruby on Rails and Django did not introduce any revolutionary database technology, nor did they represent a leap forward in computer science. They leveraged a very different technology, application design principals, and fostered a leap forward in the developer experience.

I see the same challenges in the generative AI landscape today as post-dot-com web development: flush with technology, pale with structure. Instructor is a library that monkey patches OpenAI’s sdk to enforce response classes. Instructor released an ORM integration last week that allows database objects and LLM objects to be swapped directly. The project is tightly coupled to the OpenAI product, and I don’t see the ORM integration evolving past a novelty - a service generating db objects in a vaccuum is kind of pointless, and I think you’ll end up wrapping the classes and basically building them into oddly shaped tools calls to get any real value from it. But the idea does spark thoughts around what LLM software could look like, what form the abstractions should take on within a mature framework. This is the question Software Developers should all be asking right now, and it is a question the “old programmers” may be uniquely able to answer.

What’s out there

The crop of generative AI OSS projects fall largely into one of two categories: mostly academic or mostly inexperienced.

The mostly academic projects are driven by (no surprise here) data-focused researchers in academia. Projects like MemGPT are filled with technically brilliant concepts and truly revolutionary solutions to computing problems. What they lack is design patterns, implementation strategy, or any sense of a cohesive developer API. These are typically pseudocode-that-runs, a whiteboard thought exercise in how a solution might work squeezed into production. MemGPT is not a framework or a library, it is an exciting proof-of-concept.

The mostly inexperienced projects are motivated, fast-moving, and entirely focused on the wrong shiny objects. These are very high-profile, generally VC backed endeavors that tout the number of “tool integrations” - i.e. how many API wrappers they have shoehorned into the code base - as the benchmark by which they compete. This might make sense many cycles in the future, if the core project capabilities are sound bedrock (similar to the 5,000 integrations in the django-pakages registry). But with their current trajectory, I don’t foresee most of these projects ever reaching maturity. The contributors seem most impressed with what “coding can do,” focused on the novelty of giving an LLM access to Jira, less interested in ensuring that the framework produces robust, polished, reliable, and ultimately useful outcomes. The rush by countless gen AI projects to create a novel prompt templating solution is a fantastic example of this. If there is one universally solved problem in nearly every programming language, it is “how can I generate dynamic text within templated outputs?” Prompt rendering is significantly less complex than compiling valid SQL or HTML from partials, yet many of these libraries chose to roll their own or 😱 inject raw user inputs directly into the code. Understanding the difference between an unsolved problem and a problem you personally have not yet solved is a critical skill in Engineering, something that is painfully lacking in the open-source gen AI ecosystem today.

Between the two, I believe the opportunity moment for seasoned Developers lies within transforming the mostly academic projects first. These projects need veteran programmers to think through what a framework for this technology should look like, and execute the complex meta-programming that will make these tools accessible. Architecture decisions need to be guided by those that can consider the entire lifespan of the software and the whole of the developer experience, based on hard-won knowledge. The next true change agents in the gen AI space will be David Heinemeier Hanssons, not an Alan Turings. At the same time, evolution does not happen in a bubble and there will come a time (sooner than most think I believe) when the luster of the mostly inexperienced projects will collapse on their poorly built foundations; here too, the sage Engineers who are no strangers to righting the path of a lost ship will find their place.

A perfect storm of opportunity

But why “old programmers”? What makes this demand for framework-building, design-pattern-minded Software Engineers the exclusive domain of the graybeards? AI is supposedly a young person’s game, and only a few years ago we were informed that an impending army of bootcamp grads and CompSci majors were coming to commodize the field. How can it be argued that the next faces of change in AI look more like the cast of RED than Silicon Valley?

First, meta-programming is hard. Really hard. It is the difference between knowing to use user.update(name='dave') when you want to save a name, and understanding the implications and side effects an update() function must address to work as expected. Most CS and boot camp grads are not meta-programmers; they enjoy a surface understanding of the tools they use, and are happy with a business-focused career progression; they will not go on to become “old programmers.” Instead, they graduate to Sysadmins and District Managers and CIOs, people that “used to code.” There is very little chance that these people will strike out to change the gen AI landscape any time soon.

Secondly, I believe that there would be a glut of hyper-talented, reasonably experienced, lightly seasoned younger Software Engineers on the market today, had it not been for (F/M)AANG and their big tech ilk swooping in like the spacecraft from WAL-E to pacify and infantalize huge swaths of dev talent. Old programmers owe their good fortune, in part, to the opulent insanity of the last decade; every excess Engineer Facebook hired only to keep them off the market, rotting without work, is one less seasoned programmer on the market today. Junior devs that came of age in the 20/80 work/life “balance” practiced by far too many tech teams are still junior devs today, because that lack of seat time has translated into a lack of skills progression. That is not to say that no remarkably bright, highly skilled 26-year-olds escaped from Google with more knowledge and experience than I ever will posses - some absolutely did. But there are far fewer of them than should be, given the spike in popularity of CS degrees over the last decade, and the proliferation of boot camps, online programs, and free resources available to all.

If you are an old programmer - the kind of salt that still remembers what the letters in SOLID stand for, that kind that once had Patterns of Enterprise Application Architecture on your nightstand, the kind that actually understands DNS - then this is your call to service. Put down the world’s greatest Regional Engineering VP mug, dust off the keyboard, and jump into the fray of Artificial Intelligence Software Design where your skills are most needed. Help usher in the next generation of great framework design, and secure your place in a rapidly approaching future while you are at it. The opulent software world of past years needed software managers-of-managers, the AI-driven enterprises of tomorrow may not.

the old motorcycle salesman

There is one more story to learn from in my motorcycle-selling history. An “old salesperson” who had been slinging bikes since the early 1980’s - let’s call him Rick. Rick benefited from the bubble as much as the next, purchasing a modest house and trading in his run-down Honda Civic for a new model. There came a point where it seemed like everyone was getting into the motorcycle selling game - realtors, car salespeople, even stock brokers were moonlighting with us. Rick was frustrated with this crowd - they knew nothing about the bikes they were selling, and they had no investment in the motorcycle community. After the crash, Rick left his sales manager role and went back to selling bikes. He had a little book that he would make calls from every day, and somehow a handful of customers would come in asking for him. When asked, he explained - it was an old Zig Ziggler sales strategy; every time he sold a motorcycle to a person he enjoyed working with, he would write down that person’s phone number and birthday in the book. Over a span of 20 years he filled the book until every day provided him a list of calls to make, people he already knew he liked - and almost every day’s list provided at least one customer who needed a bike, or knew someone that did. Rick the “old salesman” was there in the before-times. He was there for the crazy excess, and he reaped the benefits (within reason). He was there for the Indian summer, and smartly re-positioned himself while the rest squandered the opportunity. He comfortably settled into the new normal of motorcycle sales, which had no place for the rest of us (including myself).

The motorcycle industry is better with guys like Rick selling bikes, and Artificial Intelligence Software will be better with the guidance and grit of old programmers.