Thought Flow

Tag: programming

  • Code no more,more code than ever

    The speed of progress in the AI space right now is truly staggering. I don’t really write much code anymore, but instead direct an AI to do it for me.

    It felt odd at first, but now it does not really bother me anymore. I guess solving problems and developing solutions, architecture, design etc. is what I find most interesting anyway, so the code was a means to an end, and this part is now faster than ever to get out the door.

    Besides the stuff I do on the day job, I have been working on two small projects on the side that are ready for the world now:

    FitKeep is a backup and basic analysis tool for fitness data, specifically .fit files right now. It connects with Garmin, and I hope to Coros support soon.

    Is It Warm answers a question I often ask myself. “It sure feels unusually warm today, but is it really?” or perhaps the shorter “is it warm today?”

    Both projects were primarily created for myself. I have no business plan for them, and anyone other than me using them would just be a delightful bonus.

    Besides these new sites, I also recently modernized both Photo Amaze (Python 2.7 to 3.14) and OTB (Create React App to Next.js). I didn’t have the time and energy for this before AI made a lot of the grunt work possible to do, which is really nice.

    It’s difficult to say what the commoditization of code means for the software industry and me personally. I have opinions… but well, I’m not going to share them here, because I have been wrong about the future before, especially when it comes to AI.

  • Best of 2016

    Here are a few things that I liked in 2016, written in traditional popular culture best-of style, with a small code-related item added to the mix. I did not spend a lot of time thinking about these so the list is true to the name of this blog: Thought Flow.

    A bag of GIF

    I see quite a few GIFs every day. We use them in our team communication all the time. In fact, my first idea for this post was to write a “GIF awards”. I usually forget about them pretty quickly, but there was one that stayed with me:

    I could just have that on repeat all day! The way I’m showing it here is technically a video, but the lines between GIF and video are blurred these days :-) Here is the Original GIF.

    Flow TV is Dead (?)

    It almost seemed like we saw the final nail in the coffin for traditional TV viewing in 2016. On the streaming services however, there were many good shows to watch. The pool of choices for this award is limited, because we do not watch that many shows in our little household. The winner is pretty clear though: Stranger Things takes the prize for being original-ish, mystical-ish, sci-fi-ish, monster-ish and a few other -ish’es.

    Runner-ups:

    • The Must See: Game of Thrones, Season 6
    • The Feel Good Super Hero: The Flash, Season 2
    • The Surprisingly Engrossing But Not Highly Advertised: The OA
    • The Slow-Moving Yet Action-Packed: Marvel’s Luke Cage
    • The How The Hell Did They Pull This Off: Rick and Morty

    Language is Programming

    When people talk about “language”, I do not only think about people and nationality, I think about robots, code and programming languages. In this not-well-defined-category, the winner is Go. It deserves an award for taking me on a new journey from JavaScript la-la land, where the streets are paved with forgiveness and singular threads, to the Go nuts land of structure and order, where channels flow securely, but only when you make them.

    Sidenote: This award is a bit unfair to other languages, because Go was the only actual language that I dabbled with for a prolonged amount of time. So let me just mention Python here, because Python is the best, and nothing beats Python.

    Gamers have it good

    According to Steam Spy, 38% of all games on Steam were released in 2016. Gamers have it good. There is such a big variety and selection of games that it is sometimes overwhelming to chose something to play. I got to play a lot of games this year, and it is difficult to find a “best game” for 2016. There can only be one winner though, so that winner will be Firewatch. With its excellent and touching narrative, Firewatch really hit a home run this year.

    Runner-ups (some came out before 2016 but I didn’t play them until 2016):

    • Kill Some Aliens, or: How I Learned To Stop Worrying And Use The Mind Control: XCOM2
    • Existentialism… I got it but I don’t get it: The Talos Principle and SOMA
    • Could Have Been A Huge Hit: The Turing Test
    • Ghost Stories By The Campfire: Oxenfree and Kentucky Route Zero
    • Misanthropy, Inc: Plague Inc: Evolved
    • “Did You See That Hair?!”: Rise of the Tomb Raider
    • Underground Volcano Dance Party: Samarost 3
    • A Piece of Master: INSIDE

    Post-X, Where X Equals Rock

    I do not know whether post-rock as a musical genre is growing or not, but there were a lot of good post-rock albums coming out in 2016. In fact, if I had to name my top five albums of 2016 in general, they would all kind of fall into the post-rock or guitar-driven ambient genres. I am going to name two winners of this category, because I have binge-listened to both of them: Versus by pg.lost and The Ever Shifting Fields by Seas of Years.

    Bookworm

    No best-of list would be complete without books, would it? However, a bookworm, I am not, and I almost never, ever read new books that come out throughout the year. So let me just mention the one book that I did read in 2016 which was really good: Ready Player One (came out in 2011) is an excellent mix of science fiction and popular culture, with a hint of dystopia thrown into the mix as well.

    That’s it for the 2016 best-of list. The topics ended up being quite traditional didn’t they? That’s ok. I will be blogging about another, slightly deeper subject another time. Until then, Happy New Year!

  • Fractals revisited

    FractalGoing through old code can be fun and educational. While updating my website, I took an extra look at some of my featured code. When I came across my simple fractal simulations on the <canvas> element, I was quite surprised to see how much I violated the Don’t Repeat Yourself (DRY) principle. The three simulations share more than 80% of the same code but they were each defined in separate files where all the code was repeated. The performance of the simulation had bothered me earlier, so I decided to take a look at the code and did the following:

    • Consolidate the three simulation files into a single file.
    • Optimize the animation loop.

    It was a fun little evening project to refactor some old code. There’s still some work that could be done, like removing the hardcoded dependency of the canvas element with a specific ID, but for a little showcase like this, I do not want to bother too much about that.

    By the way, the code is online.

  • Open-sourcing the past

    While studying at the University of Oregon, I worked as a teaching assistant in three different computer science courses. One of them was CIS 323 Data Structures Lab but this course was a bit special because it had its own course number and I was teaching it almost on my own.

    It was quite a roller coaster ride… 1

    Anyway, throughout the course, we implemented some classic and often used data structures and algorithms in various forms. In my opinion, the most notable data structure we implemented was a fairly new balanced tree data structure called the left-leaning red-black tree (LLRB), invented by Robert Sedgewick in 2008. Back in the beginning of 2010, I could not find any publicly available C++ implementation of the LLRB tree 2 which made it fun to use in class because it was very new. This means that there is a possibility that my implementation was the first-ever implementation of the LLRB tree in C++. It is a fun thought but it is not very significant, considering it is only a few lines of code, the delete operation was not implemented and it was never released. Until now.

    I recently went through some old course material and found the code. So I emailed the University of Oregon and the course supervisor and with their permission, here is the code which I might expand with a few more data structures once I have looked through the material. I have refactored the code from the original but it still has the mark of a C++ beginner. It was fun going through it again though.

  • Experimental features

    This post is about Antecons, a product recommendation engine, now part of Conversio. Antecons is no longer commercially available, but I have kept my developer diary on my website with permission.

    Yesterday, I found out exactly what it means when Google warns about their experimental App Engine features: Your code might eventually break. Let me be clear, I am not blaming Google. They give you fair warning:

    Mapreduce is an experimental, innovative, and rapidly changing new feature for Google App Engine. Unfortunately, being on the bleeding edge means that we may make backwards-incompatible changes to Mapreduce.

    I have written about my usage of the MapReduce framework earlier. Yesterday, I updated the MapReduce framework to the latest version only to see that my custom Datastore reader suddenly had stopped working and I was seeing exceptions in my MapReduce pipeline. Bummer.

    Long story short, I spent a day debugging the new code and finally got it working by:

    1. Digging through the MapReduce framework code. Hurray for open source!
    2. Dropping the idea of running FP-Growth on batches of entities and instead running the mapping function on each entity.

    That second point probably requires some explanation to really grasp and I am not sure I will be able to but maybe some pseudo-Python will help. The biggest change happened in the map-step of the Frequent Patterns MapReduce pipeline. Basically I went from this:

    def map_batch_of_transactions(batch):
    frequent_patterns = fpgrowth.run(batch)
    for p in frequent_patterns:
        yield p, p.support

    to this:

    def map_single_transaction(transaction):
    frequent_patterns = itertools.combinations(transaction, 2)
    for p in frequent_patterns:
        yield p, 1

    The MapReduce shuffler takes care of grouping together patterns with the same key so with the new method, the shuffler will have more work to do since the same patterns will be yielded more often. Let’s say we have the pattern:

    a,b (support: 4)

    Before, the shuffler would just receive:

    ('a,b', 4)

    but now it will receive:

    ('a,b', 1)
('a,b', 1)
('a,b', 1)
('a,b', 1)

    On the other hand, FP-growth does not have to run so the map-step of the pipeline has more predictable performance characteristics. It remains to be seen if the change has significant impact on the entire MapReduce process. I am currently testing this.

    So anyway, the whole point of this post was: If a feature is experimental, watch out. Sounds obvious right? Well…