Neurons spike back

Neurons Spike Back (https://neurovenge.antonomase.fr/) was featured in the latest data science weekly newsletter. I would normally pass on such long articles, but the history of AI is interesting, so I gave it a shot. Reading the paper felt like a marathon, and I only completed it through sheer force of will, lots of coffee, and because it is cold and raining outside.

The article is very difficult to read (at least for me), not because it is filled with theory, but because the language is dry and academic, and it tries to condense decades of history around the research of AI into a fairly short (given the topic) article that discusses two opposing sides of research and thought: connectionist and symbolic AI.

Despite my warning above, if you have the patience, I found it to be a fairly good overview of how we ended up where we are today with deep learning dominating the state of the art for AI in many fields.

I found it particularly interesting to learn about the Mark I, a hardware neural network constructed during the 1960’s for simple object recognition. It is a good reminder that the concepts we are using today have been around for a very long time, and I often find that knowing a bit of the history behind them help understand what we are doing in the present.

Reinforcement learning

I have been looking into a machine learning technique called reinforcement learning (RL) lately. This was on my TODO for a while, and I must say, this field is incredibly exciting! I played around with some OpenAI Gym environments and re-implemented two RL algorithms mostly based on code I found from other authors.

After spending many hours on this, I can still only get my algorithm to solve the Cartpole problem, where the goal is to balance a pole on a moving cart (video below). I haven’t cracked the nut on a continuous action problem like Pendulum, where the goal is to swing the pendulum into an upright position and keep it there (video below).

Anyway, here is my implementation of the RL algorithms. Perhaps it will be useful for someone :-)

Dig the Data, StoreGrader edition

A new Dig the Data was published yesterday. It has some data insights from StoreGrader which is an app I have been working on for a while now.

For this edition of Dig the Data, I wanted to create a nice looking interactive infographic, and I wanted to combine both static and interactive elements. My previous Dig the Data visualization was quite minimal, but had full interactivity. However, it lacked a bit of the feeling of “niceness” that some static graphics can provide (as well as the magic touch of a designer, which I am not). A good example of this “niceness” is the first Dig the Data, where the entire visualization is a static image created by my colleague Julia.

This time, I teamed up with Maria to create a visualization that combines both static insights (with a bit of animation) as well as interactive graphs to explore.

I am very pleased with the result, and you can check out the post here.