Apache Beam MongoDB reader for Python

The Apache Beam SDK for Python is currently lacking some of the transforms found in the Java SDK. I created a very minimal example of an Apache Beam MongoDB read transform for Python that might be useful for someone else looking for an answer.

I will update this post in the future if/when the Apache maintainers include support for MongoDB in the SDK. I know I could contribute to the project directly, but I don’t have time for it right now unfortunately :-)

I hope you learned something from that

Start of Valdresflye

At the beginning of 2016, I was hoping for a more “creative 2016” and had internalized a few secret New Year Resolutions: Write a blog post at least once per month, release a music track every two months, learn to draw, read more books, make weekly programming projects, and a few others that I cannot remember.

Not surprisingly, none of this happened. With so many goals, most of them unclear, I was bound to disappoint myself.

Part of the reason is that my creativity is limited, and most of it is spent on work-related endeavors. For example, I would not have learned the Go language if I had not used it for work-related projects. Fortunately, I have a job that allows me to grow, but unfortunately, it also means that I often do not have enough extra creative energy to pursue some hobbies outside work.

For example, regarding music, I only managed to release a single track. This feels like a failure compared to my secret goal of releasing something every two months. In fact, it was already a major letdown half-way through the year when I had produced nothing.

And let’s not talk about all the other things I wanted to do…

I have been struggling a lot with creative willpower throughout 2016. I often felt like I was “wasting my time” watching tv or playing a video game, but without having the will or capacity to actually do something about it. For certain activities, it is so easy to say “just start it”, but much harder to actually do it. I still believe that forming habits is a good first step to get stuff done, but I think what I learned is that I have limited resources to do so.

In the end — and with the help of my wife — I have concluded that it is OK to not always do everything at once and to be happy with things in life that do not require as much creative or mental energy as learning something new or improving an existing skill.

This is one of my top learning experiences of 2016: It is not possible to do everything, and that is OK. So here is my only New Year Resolution for 2017: Accept what I do, not what I want to do, and let the year be what it becomes.

Best of 2016

Here are a few things that I liked in 2016, written in traditional popular culture best-of style, with a small code-related item added to the mix. I did not spend a lot of time thinking about these so the list is true to the name of this blog: Thought Flow.

A bag of GIF

I see quite a few GIFs every day. We use them in our team communication all the time. In fact, my first idea for this post was to write a “GIF awards”. I usually forget about them pretty quickly, but there was one that stayed with me:

I could just have that on repeat all day! The way I’m showing it here is technically a video, but the lines between GIF and video are blurred these days :-) Here is the Original GIF.

Flow TV is Dead (?)

It almost seemed like we saw the final nail in the coffin for traditional TV viewing in 2016. On the streaming services however, there were many good shows to watch. The pool of choices for this award is limited, because we do not watch that many shows in our little household. The winner is pretty clear though: Stranger Things takes the prize for being original-ish, mystical-ish, sci-fi-ish, monster-ish and a few other -ish’es.

Runner-ups:

  • The Must See: Game of Thrones, Season 6
  • The Feel Good Super Hero: The Flash, Season 2
  • The Surprisingly Engrossing But Not Highly Advertised: The OA
  • The Slow-Moving Yet Action-Packed: Marvel’s Luke Cage
  • The How The Hell Did They Pull This Off: Rick and Morty

Language is Programming

When people talk about “language”, I do not only think about people and nationality, I think about robots, code and programming languages. In this not-well-defined-category, the winner is Go. It deserves an award for taking me on a new journey from JavaScript la-la land, where the streets are paved with forgiveness and singular threads, to the Go nuts land of structure and order, where channels flow securely, but only when you make them.

Sidenote: This award is a bit unfair to other languages, because Go was the only actual language that I dabbled with for a prolonged amount of time. So let me just mention Python here, because Python is the best, and nothing beats Python.

Gamers have it good

According to Steam Spy, 38% of all games on Steam were released in 2016. Gamers have it good. There is such a big variety and selection of games that it is sometimes overwhelming to chose something to play. I got to play a lot of games this year, and it is difficult to find a “best game” for 2016. There can only be one winner though, so that winner will be Firewatch. With its excellent and touching narrative, Firewatch really hit a home run this year.

Runner-ups (some came out before 2016 but I didn’t play them until 2016):

  • Kill Some Aliens, or: How I Learned To Stop Worrying And Use The Mind Control: XCOM2
  • Existentialism… I got it but I don’t get it: The Talos Principle and SOMA
  • Could Have Been A Huge Hit: The Turing Test
  • Ghost Stories By The Campfire: Oxenfree and Kentucky Route Zero
  • Misanthropy, Inc: Plague Inc: Evolved
  • “Did You See That Hair?!”: Rise of the Tomb Raider
  • Underground Volcano Dance Party: Samarost 3
  • A Piece of Master: INSIDE

Post-X, Where X Equals Rock

I do not know whether post-rock as a musical genre is growing or not, but there were a lot of good post-rock albums coming out in 2016. In fact, if I had to name my top five albums of 2016 in general, they would all kind of fall into the post-rock or guitar-driven ambient genres. I am going to name two winners of this category, because I have binge-listened to both of them: Versus by pg.lost and The Ever Shifting Fields by Seas of Years.

Bookworm

No best-of list would be complete without books, would it? However, a bookworm, I am not, and I almost never, ever read new books that come out throughout the year. So let me just mention the one book that I did read in 2016 which was really good: Ready Player One (came out in 2011) is an excellent mix of science fiction and popular culture, with a hint of dystopia thrown into the mix as well.

That’s it for the 2016 best-of list. The topics ended up being quite traditional didn’t they? That’s ok. I will be blogging about another, slightly deeper subject another time. Until then, Happy New Year!

Post-Truth

Day 15 #Truth
Day 15 #Truth by mishey_mouse (CC-BY-NC)

Word Of The Year is without a doubt “Post-Truth”1. In my interpretation, it perfectly captures two observations about the current world:

  1. Truth largely does not matter anymore. We have moved beyond truth, we are post truth.
  2. Truth can be created after (post) a statement is made — it does not have to be true originally.

Post-truth has come up a lot lately. It is often mentioned in association with Brexit and Donald Trump’s election as president, but I do not have anything significant to add to these specific debates on this blog.

But I will say this: Post-truth is dangerous… It is dangerous, because it deceives. It is dangerous, because it is used to incite hatred and create injustice. It is dangerous, because it challenges the fundamentals of civil discourse: Honesty, respect and integrity.

This year, we have mostly seen post-truth in public politics. However, I fear that it is not a temporary phenomenon, but a symptom of a larger disease, slowly infecting our society. On a large scale, the alt-right movement is gaining traction, fueled by fake news and hatred. On a smaller scale, it has become trendy to not be politically correct.

The result of post-truth is quite dark: Thumbs up for telling lies, they will become true eventually. Thumbs up for ignoring real facts, they can be created.

I am not sure that we can reverse the rising trend of post-truth. That ship has sailed, and we have to get used to the era of post-truth. However, I do think that being aware of the problem enables us to do something as individuals, even on a small scale.

On my part, I will try to be more critical towards my news sources and politicians, I will try to be respectful and honest in my discourse, and I will be politically correct to counter-balance those that are not. This is the best I can do, and I know that I am not alone. That is a comforting thought.


  1. Actually, it seems that Oxford Dictionaries named post-truth word of the year 2016 back in November. I honestly did not know that before I started writing this blog post. Here is how post-truth is defined in the dictionary:

    Relating to or denoting circumstances in which objective facts are less influential in shaping public opinion than appeals to emotion and personal belief Oxford English Dictionary online