The Past as Preview, Part 1: Foundations of Sand and The Future Decline of AWS

It’s been three months since my last entry, something that I did not expect.  In the next entry, I will update you on some of the exciting recent changes and why it took me three months to complete an entry that I thought would be done in a week, but for now, I’d like to look to the past as a preview of the future. 

Study the past if you seek to define the future – Confucius

That quote is the positive version of Harvard philosopher professor George Santayana’s overquoted comment on history repeating itself.   I prefer the positive version because it is the original and because I believe that in the face of great difficulties, we need to create. The past is useful to look to not only for the instruction of what to avoid but also to motivate where to go.

I will address the past as a preview of the future in this and the next two entries, addressing “platforms,” software languages, and geographical locations where great “scientific leaps” often occur. With that said, let’s go back and reflect a bit on the dominant technology platform of the past – IBM.

The Past as Preview of the Future

In many ways, this COVID-impacted time reminds me of the period that marked the rise of the current financial infrastructure.  In the 1950s, computer scientists began attempting to define the computerized infrastructure of the modern financial system, but it took the 1973-1975 Recession to create a situation favorable for groups to start embracing the innovations that had been patiently cultivated by a small group of programmers based at UC Berkeley. 

While every investment bank on the East Coast fought to control the future, all working with an IBM hardware and software platform, a rag-tag group of radical programmers at Berkeley began working to define the future.  Rather than working with big banks, they looked to the asset owners like insurance companies and the rare West Coast bank or brokerage that were considered outsiders to the Eastern financial elite. Those Berkeleyites had done the most revolutionary of things, especially in finance: they openly collaborated and even made their technologies available to the masses, an early version of what we now call open-source.  Even more radical, they did not work with the IBM platform. It was simply too expensive and it didn’t give them what they wanted or needed. Instead, they worked with various other technology firms, mostly small Bay Area-based companies and academic groups, and cobbled together better solutions than the monolithic solutions that one got from IBM.

Keep in mind, this was when every IT manager simply bought IBM, just like they do today with AWS.  The statement was, in fact, “you never get fired for choosing IBM.”  Thus, the people looking to limit their own downside always chose IBM. That said, this choice taken to limit the individual downside to the IT professionals inevitably limited the upside of what those organizations could achieve. So while every investment bank was working on the IBM platform to create their own proprietary systems, bit by bit, the Berkeleyites gained ground.  That was what I refer to as the “Risk Revolution.”

In the 1960s and 1970s, the saying among IT professionals was “You never get fired for choosing IBM.” That said, you don’t start a revolution by going with the establishment!

Those Berkeley-created models are now used by over 70% of the world’s assets to define and control risk.  My dissertation advisor joined Morgan Stanley and was among the people who convinced Morgan Stanley leadership that the battle was over and the Eastern banks all lost.  He convinced them to buy the victor and become its champion.  That was the genesis of what is today a $30 billion public company known to the world as MSCI, formerly “Morgan Stanley Capital International.”  It is not random that MSCI has been our biggest supporter since the ULISSES Project began.  That was by design. In many ways that relationship began to flourish when my good friend and Advisory Board member John Regino came to understand that the ULISSES Project was about completing the Revolution that had started so many years ago at Berkeley.  In short, the ULISSES Project is about what I refer to as “the Return Revolution,” or the completion of the Risk Revolution.

The Return Revolution (or Forecasting Revolution)

We always say that investment is about “alpha and beta,” or in more familiar terms, “return and the associated risk.”   The Berkeleyites, who defined the current financial system’s risk infrastructure, started by addressing systems that forecasted alpha or future return but they found that the world was not ready technologically or intellectually for such systems.  As a result, after they presented their results to the world, one of their greatest spokesmen, Richard Mattessich, tied a neat little bow on that earlier “return-based work” in 1978 with his magnum opus entitled Instrumental Reasoning and Systems Methodology and the researchers then focused on rallying under the banner of “Barr’s Bionic Betas” or what the investment world could understand and technology could address at the time. That was the beginning of the Risk Revolution. It had been years in the making, but it was won within 36 months once all the parties rallied together under the Bionic Beta Banner.  

In his great work, written explicitly for future generations, Mattessich explained that the Berkleyites were using finance to demonstrate a universal technology, a system to address forecasting and proper statistical inference in every aspect, from physical systems to biological systems.  Economics and finance were simply the examples that could provide the fastest and most powerful (through the money generated) feedback loops.  While this notion might seem strange, those brilliant researchers were hijacking the capitalist system to produce universal technologies that could benefit humanity.   Before scoffing and dismissing this, one should consider that Mattessich was the father of the electronic spreadsheet, and he clearly stated that the spreadsheet was just a taste of things to come.  Reflect on this for a moment—the spreadsheet, the first “killer app” of the PC age, was foreshadowing!

Universal Technologies

For those that doubt that we are addressing universal technologies in the ULISSES Project, I recount that only a few years ago, I sat down with Lee Hood for over five hours at the Scientific Computing and Imaging Institute at the University of Utah and discussed this very issue.  No, I did not kidnap Lee. It was quite the opposite when I was pulled away by university administrators from our first-generation Joshua servers to talk to Lee and tell him what we were doing.  After I described our project and discussed the prototype systems we’d developed with Dell’s support, Lee would not let me return to my work.  He insisted I be his companion for the next two days. 

Lee was not only intrigued by what I explained, but he invited me to come to Seattle, stay at his home, and then address the leadership of his Institute for Systems Biology.  I accepted, and after spending the evening with Lee, I spent the next day addressing his institute.   The presentation lasted for 12-hours and then continued into the night at his home.  Lee was intrigued by what we had done and he proposed that we merge the ULISSES Project into a joint research institute. I considered such an offer from a living legend as a great compliment, but nonetheless, it was something I declined because I said we were not ready yet for such a move. I said that we’d not cracked finance yet and we’d learn so much more by addressing those areas I know so well first.  Once we got those systems completed, I told him it would be relatively easy to apply those same technologies to biology and other areas, not unlike how spreadsheets are now used by everyone for pretty much anything.  In short, I said in building universal technologies, one must be careful in choosing the initial area of application. Something I’d learned from studying history (thanks Confucius).

Foundations built on the Sandy Soil of AWS

Lee had not just wanted to form a joint institute, but he was also interested in using our innovations in his newest commercial endeavor, Arivale, a company that had the objective of merging all personal data and, in effect, creating forecasts of personal health.   Lee was an innovator who’d literally created the gene sequencer, helped found dozens of successful companies, and he’d raised $50 million for his new venture, Arivale, which he was sure would change the world. When in Seattle, I discussed with his staff how they had built their systems, and I gave them the unfortunate news that their systems could not be salvaged because they were built on the wrong technological foundations, and due to that, the whole systems would have to be rebuilt from the ground up.  This was a hard message, but one that I well knew from experience.

Lee Hood and party at Arivale built on AWS. It was free and they got what they paid for, quite simply an infrastructure that could not accommodate complex systems forecasting.

The main problem was that Lee and his Institute had succumbed to the allure of Amazon Web Services (AWS) because Amazon had given them free use of AWS in order to get them to build a medical-based information ecosystem.  The people working with Lee figured they could not go wrong with “the standard,” not unlike earlier generations of IT professionals had thought about IBM. They could not have been more incorrect. In that sense, AWS has become a foundation of sand for people working on complex systems problems, I knew this because I too had built a house on a foundation of sand before.

I did this in 2010 when AWS was relatively young. I had played with it when it launched years before, I was intrigued and so in 2010 I built my entire forecasting systems on AWS. By 2013 I had not only pushed the limits of AWS but understood the implications for the future of their architecture and business plan.  With this insight, I convinced Dell to begin working with the ULISSES Project in 2017 to build a prototype for an alternative infrastructure.   After we finished the prototype systems to accommodate the coming Return Revolution (those same systems I was torn away from to talk to Lee Hood), our ULISSES Advisory Board member assigned to us from Dell joined AWS to evangelize such systems.  After spending 18-months trying to convince Amazon to address the issue properly, he left mid-2019, right around the time Lee and Arivale started to reap their own bitter AWS harvest.

The Innovators’ Dilemma and AWS

In short, Amazon leadership rejected our former Board member’s evidence and perspective because AWS was making too much money providing generic infrastructure that would never be able to handle complex computing that integrated systems processing like weather, financial, or human health forecasting requires.  They decided this in the face of two Turing Award winners’ 2017 acceptance speech which laid out a direction for the future, evidence from MIT’s Computer Science and Artificial Intelligence (CSAIL) Lab, and our own ULISSES Project evidence. It was precisely the situation described by Clayton Christensen in his classic work The Innovators Dilemma.  In short, companies allocate capital based mainly on profits and profit margins, and that is heavily influenced by the current capital (and knowledge) base. As a result, they miss the impact of innovations, which are usually applied to lower margin areas by other companies first.  Those companies eventually rise as that technology matures and destroy the incumbent’s lucrative business.  

As weird as it may seem right now as we face AWS global dominance, the seeds of the destruction of the world’s largest cloud services firm have already been sown in the sandy soil.  I consider the first of those seeds to have sprouted when Lee’s business Arivale folded at the end of April 2019.  One month later our former Board member left AWS. For many, that may not seem like a big thing, but Arivale was a great idea, started by a proven entrepreneur, and it was very well-funded—it was just built on a free foundation of sand. 

Leave a Reply

Your email address will not be published. Required fields are marked *