AI computing requirements

Whenever there is a new announcement or breakthrough with AI, it always strikes me how out of reach the results would be to replicate for individuals and small organizations. Machine learning algorithms, and especially deep learning with neural networks, are often so computationally expensive that they are infeasible to run without immense computing power.

As an example, OpenAI Five (OpenAI’s Dota 2 playing bot) used 128,000 CPUs and 256 GPUs which trained continuously for several months:

In total, the current version of OpenAI Five has consumed 800 petaflop/s-days and experienced about 45,000 years of Dota self-play over 10 realtime months.

OpenAI blog post “How to Train Your OpenAI Five”

Running a collection of more than a hundred thousand CPUs and hundreds of GPUs for ten months would cost several million dollars without discounts. Needless to say, a hobbyist such as myself would never be able to replicate those results. Cutting edge AI research like this has an implicit disclaimer: “Don’t try this at home”.

Even on a smaller scale, it is not always possible to run machine learning algorithms without certain trade-offs. I can sort a list of a million numbers in less than a second, and even re-compile a fairly complex web application in a few seconds, but training a lyrics-generating neural network on less than three thousand songs takes several hours to complete.

Although a comparison between number sorting and machine learning seems a bit silly, I wonder if we will ever see a huge reduction in computational complexity, similar to going from an algorithm like bubble sort to quicksort.1

Perhaps it is not fair to expect to be able to replicate the results of a cutting edge research institution such as OpenAI. Dota 2 is a very complex game, and reinforcement learning is an area of research that is developing fast. But even OpenAI acknowledges that recent improvements to their OpenAI Five bot are primarily due to increases in available computing power:

OpenAI Five’s victories on Saturday, as compared to its losses at The International 2018, are due to a major change: 8x more training compute. In many previous phases of the project, we’d drive further progress by increasing our training scale.

OpenAI blog post “How to Train Your OpenAI Five”

It feels slightly unnerving to see that the potential AI technologies of the future are currently only within reach of a few companies with access to near-unlimited resources. On the other hand, the fact that we need to throw so many computers at mastering a game like Dota should be comforting for those with gloomy visions of the future :-)

Reinforcement learning

I have been looking into a machine learning technique called reinforcement learning (RL) lately. This was on my TODO for a while, and I must say, this field is incredibly exciting! I played around with some OpenAI Gym environments and re-implemented two RL algorithms mostly based on code I found from other authors.

After spending many hours on this, I can still only get my algorithm to solve the Cartpole problem, where the goal is to balance a pole on a moving cart (video below). I haven’t cracked the nut on a continuous action problem like Pendulum, where the goal is to swing the pendulum into an upright position and keep it there (video below).

Anyway, here is my implementation of the RL algorithms. Perhaps it will be useful for someone :-)

A bank transaction analysis tool for the browser

Screenshot from Off The Books showing most common expenses in 2018

This graph shows my four most common expense categories during 2018 and how they changed from month to month. It is a screenshot from one of the charts in OTB, a bank transaction analysis tool that I have been working on for a while.

OTB was created to fulfill my need for seeing what expense categories I spend my money on. Although a lot of apps can do this, I had some special use cases that were not quite being solved by existing solutions. On top of that, it concerned me a bit to hand over my bank transactions to a 3rd party, even though some financial apps are regulated and have good security.

Privacy might actually be the main feature of OTB at this point. All the data (transactions, categories, etc.) are kept in the browser only, instead of storing the data in the cloud. It is thus impossible for the data to leak, unless the browser itself is compromised.

I have been using OTB regularly for a few months now. I have added and categorized 3840 bank transactions from two different banks in two different currencies (DKK/SEK), stretching back 3-6 years. A simple machine learning algorithm helps select categories, and when I add new transactions (which I do once or twice per month), I usually have to categorize 3-7 transactions by hand, and the rest (anywhere between 20 and 100 transactions) are correctly guessed by the algorithm, because my purchasing patterns are quite regular from month to month.

To be honest, the app is a bit messy, and not at all user-friendly, but I am happy with the outcome of the project so far, since it has actually started to become useful to me. For example, the chart at the beginning of this post shows, among other things, that I spent more on eating out in February and June than I spent on groceries. This is the kind of insight I was hoping to get from the app when I started working on it, and there is a lot of room for improvement.

I could go into a lot more detail about why I created OTB, what the limitations and trade-offs were in the design, and what I plan hope to do next, but I will save this for another time :-)

For now, feel free to check out OTB or take a look at the source code.