Defense Against the DARQ Arts, Optimization: Part 3 – My Optimization Odyssey, the Mormon Trail

My Optimization Odyssey continued in Utah, but not without a temptation to go back to Cali.  In many ways, I was doing a reverse trek.   You see, Wallace Stegner, the legendary founder of the creative writing program at Stanford started out in Utah.  He wrote tons of impactful books, winning lots of awards, and to honor his “Utah roots” he wrote a book about the Mormon Trail (which ended in Northern California).  Stegner then left Utah himself for the Bay Area where he mentored Kesey at Stanford, guiding him in his account of working as an orderly at a mental institution in Oregon. That book eventually became One Flew Over the Cuckoo’s Nest, an international bestseller, inspiring a multiple Academy Award-winning movie and helping define actors such as Jack Nicholson. In high school, I loved Kesey’s masterpiece and creative writing and this, sparked by readings and hangouts at Powell’s, made Stanford almost irresistable. 

The saving grace was when I was 17 and Stanford reached out to me, my family was dead broke due to my dad’s “environmental crusade.” My dad demanded I go to Stanford or an Oregon university, but I had another idea. I’d become a Mormon when I was 14 (Eldridge Cleaver, one of the leaders of the Black Panther Party, and I were both introduced to Mormonism when we were on probation and both of us later became Mormons after being released from probation). BYU had offered me a full four-year scholarship and an opportunity to work on campus, so I accepted and went to the Mormon homeland of Utah.

BYU First Round – The B-School Computer Lab

Soon after arriving in Provo, I went to work for Bob Kellett, a former defense department programmer who ran all the computers in the business school. Bob had been recruited to translate one of the first financial forecasting and investment programs ever, something written in Machine Code that he translated to Fortran.  It was amazing and years ahead of its time. As weird as it might seem now, my “programming test” when I got on campus was on the Macro language for Lotus 1-2-3. Because I could program (most in business schools can’t), I did the best in the history of the test and that’s why I got my job assigned to help supervise a computer lab in the business school that specialized in training future executives for Novell and WordPerfect.  For the next two years, I found myself tutoring students in a spreadsheet macro language and other simple things like HyperCard and databases.  While my day job was the definition of routine, it gave me spending money and the keys to the lab in which I would often pull all-nighters on my various programming projects trying to merge the technologies I’d worked on over the years.  Those nights were fueled by two BYU-banned substances, Jolt cola, and heavy metal.

Two “Ultimate Sins” at BYU kept me going late into the night. While not the Great Ozz’s best album, it did have a cover by Boris Vallejo of Conan fame that later helped inspire our Conanfest hacker get-togethers at the Tolley home where Hal and I first bonded. Math, Arnold-movies, and Video Games…all served with meat you could eat with your hands. What could be better in life?

It was there in the BYU Computer Lab that I ran into a brilliant little analysis plug-in that included linear programming (this will come into the story later at Columbia).  I was hooked and assumed that if I could just get a computer powerful enough, I could solve the work I had started on in Portland in one integrated system, but I soon came to realize that the IBM PC and Lotus 1-2-3 just couldn’t cut it, just like the Mac and HyperCard. After two years of working in the lab on these problems, I packed up my computer and went on a religious mission to Rome, Italy where I didn’t even touch a computer for two years.

Intermezzo in Italy

As weird as it will seem, it was on my mission in the Umbrian city of Perugia, where I gained more perspective on my project and started to figure out some of the things I was missing.  My mission wasn’t what you’d call a “typical mission,” with me actually being assigned to be a student for a portion of it. It was after my “initial training” that I was directly introduced to the original works of Leonardo da Vinci’s math teacher, Luca Pacioli, in the dusty libraries of the University of Perugia, the city where he used to teach and where he published the first math book in Europe since Euclid’s Elements, his Summa de Arithmetica

Made in obvious reference to Dante’s 34th Canto of the Divine Comedy, the last Canto of the Inferno where Dante and Virgil encounter Satan himself. Tell me how just a metal instrumental merits an “E” rating? Inspired obviously by the melting of brains that EVH did for Darth Vader from Planet Vulcan.

What gave me even more perspective than Pacioli was reading Dante with my great teacher and mentor Fausto Grimaldi. This last thing makes sense when one understands that minds like Leonardo, Michelangelo, Galileo, and Fermi were devoted to their Dante studies. That is an even longer story, so I will skip that for now, but it is worth mentioning that when Barr Rosenberg, who led the last revolution in finance from Berkeley, was asked how he could keep all the interplay of the markets in his mind he said something similar to what I would (and related to my Dante studies). Rather than say something like he did “mental gymnastics,” Barr said he was able to keep track of things so easily because he had learned to do so by studying Shakespeare’s “interplay of diverse interests,” something he learned in his youth from his father, a professor, and world-renowned Shakespearean scholar. Armed with my Pacioli and Dante I came back to Utah.

BYU Second Round – The NeXT Computer Lab

The day I got back to Utah from Italy, I stopped by the Math building at BYU to see what had happened when I was gone and met the head of the Statistics department, Harold “Dennis” Tolley. I spent the rest of the day with Dennis, having dinner at his home that night.  That day he made three introductions that would change my life:  The BYU NeXT Computer Lab, his oldest son and namesake (my partner Hal Tolley), and Brazilian Jiu-Jitsu.  While Hal was only a few months older than me, he was already working at Skunkworks at Novell ( eventually working and reporting to Eric Schmidt of future Google fame).  There couldn’t have been a bigger contrast than between me and Hal, that first night. I was a clean-cut, white shirt-wearing guy who’d just got back from Rome and was all smiles, and Hal was wearing a ripped t-shirt, had a pink mohawk, two dangling earrings, and a perpetual scowl.  Even though he thought of me as a “suit,” he soon came to realize that there was more inside and became my best friend and lifelong collaborator, but I get ahead of myself.

It’s the nerd in me that likes Utah Saints’ Mortal Kombat theme. I came home from Rome to be introduced to Jiu-Jitsu and advanced Bayesian Learning techniques by my parnter’s dad that merged the two in a Mortal Kombat-like game that prpogressively learned your every move. It was later applied in an embedded AI-chip Hal designed for New Holland tractors.Flawless Victory”

Every day I spent hours with Hal’s dad in the gym, discussing things like “stochastic optimization,” “Bayesian updating” and Dennis’ passion project with Hal where they were using a “Mortal Kombat”-like game on steroids to demonstrate their AI, showing it could adaptively respond to even the best players and beat them in just a few rounds starting from nothing.  While the game was interesting, I spent even longer hours in the NeXT Computer Lab (one of the few in the nation set up by Steve Jobs personally) trying to implement the ideas Dennis and I spoke about in finance and economics.  In the NeXT Lab, I saw the promise of merging and improving everything that I had pursued in Portland on the Mac in HyperCard and in the business school computer lab on the IBM via Lotus and databases, all via the NeXT integrated-oriented system with Mathematica, Lotus Improv, and Sybase. While Mathematica could deal with linear optimization routines, I found that it had huge problems dealing with the data despite Jobs’ greatest intentions with the integrated object-oriented framework.

BYU Third Round – SAS and Econometrics

In those days all of Wall Street was abuzz about the possibilities of the NeXT, with armies of analysts and programmers trying to figure out how to use Steve Job’s new invention.  I had a front-row seat and all the software I could need set up by Jobs personally so I spent the next 6 months implementing a forecasting and simulation framework integrated with an optimization routine and then wrapping it all in a customized graphical interface that I was sure would change the world.  After pushing the NeXT ecosystem to its limits, I realized that it just wasn’t there yet, and eventually so did Wall Street. With my NeXT Project not working out how I hoped, I then looked to SAS, extending my time at BYU (after all I still had the scholarship to use) to do a master’s degree before my Ph.D. studying a mix of advanced math, Renaissance Italian literature and spending my evenings grappling with Mark Schultz, a fellow refuge from Palo Alto, and my nights improving my SAS framework, doing everything I could in their IML (Interactive Matrix Language).

There could not have been a bigger difference between Columbia and UNC. I turned down Columbia three times, until finally, I relented, choosing the business mecca over the stats mecca. UNC and Duke are two of the very best statistics programs in the world, having spawned companies like SAS and Quintiles, respectively, the largest analytics software and consultancy companies in the world.

I had started talking to Ph.D. programs when I was 19 and finally, I had to take the plunge and leave BYU.  I initially accepted an offer with UNC-Chapel Hill (where Dennis’ got his Ph.D.) where I planned on working with the SAS Institute and some of the best statisticians in the world, but Columbia then came late to the game and made me an offer I couldn’t refuse to lead programming for a series of projects at the business school with Wall Street.  I was the only person in the history of UNC to turn down the scholarship they offered me, but I just couldn’t turn down Columbia as they even told me I would also have the opportunity to study Italian Literature with one of my heroes Umberto Eco, who unfortunately announced he canceled his Columbia stint weeks after I arrived in New York (I was mad about this for years).  By the time I got on the plane for New York, I had coded everything in my integrated system in SAS developing my own linear optimization routines in IML. Little did I know, I was soon going to figure out how primitive my optimization routines were.