Antecons developer diary part 1

This post is about Antecons, a product recommendation engine, now part of Conversio. Antecons is no longer commercially available, but I have kept my developer diary on my website with permission.

---

It has been a year since the first announcement of Antecons. What has happened since then? Absolutely nothing. The reason for this seems to be pretty straightforward: I have been occupied with full-time consulting and I have made a deliberate choice of working a five-day 40-hour work week. That leaves no room for extra work. However, this has changed slightly. Since October 1, I have had a four-day work week. This leaves one day to do other stuff and I have decided to continue my work on Antecons and I intend to blog about the (slow) progress.

Consider this part 1 in a series of posts about my experiences, design decisions and the current state of Antecons. First, I would like to fill in the gap in the history of Antecons from last year until now. There is more to the story than just a lack of time.

Where we left off

One year ago, I made a promise, a prototype and a teaser. The prototype is a recommendation engine that is still running on one webshop (as far as I know). It is written in C# for .NET, uses Microsoft SQL server as database and is deployed directly on the end-user’s server using a pre-compiled DLL that can be referenced from any .NET application. As I was developing the prototype, I already had a couple of basic objections to my own design:

1. It required the user to use .NET and SQL Server.
2. It required me to use Visual Studio, IIS and Windows as a development environment.
3. It required the user to integrate the component into their webshop. This included:
   1. Setting up an SQL database and tables conforming with the Antecons data format.
   2. Synchronizing webshop purchases with the Antecons database and keeping it updated.
   3. Manually running the recommendation algorithm to pre-create recommendations.
   4. Integrating Antecons into the webshop design in the appropriate places to show the pre-created recommendations for the current product being shown.

The first two points were easily fixed by deciding to replace the pre-compiled DLL component with a webservice. Using an API for Antecons, there would be no constraints on the user regarding their server software or webshop programming environment.

And thus was born the teaser, a web application written in Python and running on Google App Engine. It never turned into a webservice with an API. Instead it was a website that showed what the essence of Antecons was: A recommendation engine based on association rule mining (or market basket analysis). As it turned out, this was the easy part and implementing a simple API would have been too. But the third point I outlined above, that was the real killer.

The promise

A quote from my company website:

Metacircle has been developing a data analysis tool for some time. It is called Antecons.

And from the Antecons teaser website:

The API is currently under development…

These quotes sum up the promise I made myself (and the world): To work on Antecons. Did I succeed? No. Do I feel bad about that? Yes. What killed the project? Time and wishful thinking.

Time grew short when I started working full-time but this is only part of the explanation. As hinted above, I was also held back by the third design objection. Setting up Antecons for actual usage was, in my opinion, too difficult and too much an obstacle for widespread usage. How could Antecons be made so it would be super easy to setup, super easy to deploy and super easy to run. The ideal scenario looked like this:

1. User registers for Antecons and receives a unique User ID.
2. User inserts 10 lines of Javascript code in the header of each webshop page where Antecons should show recommendations.
3. Antecons takes care of the rest.

This is how e.g. Google Analytics works and it is quite amazing how much information it gathers and how useful this information is, based on such a simple installation. If Antecons could be as easy to install as Google Analytics, which is installed on millions of websites, there would be a huge potential and many users could benefit from it.

But the difficult questions piled up fast. For example, how would Antecons gather information on user’s sales data and how would it know where to show recommendations on a user’s webshop. The answers I came up with all lead to the same conclusion: The user would have to do some work and Antecons could not be fully automated, not even partially. This shattered my overall vision for Antecons and my head has been throwing around ideas ever since. Maybe the ideal scenario was wishful thinking but in any case, this was the end of the line. Until now.

Where we are

It has been two weeks since I started over with the Antecons project. In the next blog post, I will elaborate on what specific design obstacles that put the project on ice. This is an important pretext for what I have started working on right now. Later, I will write more about specific design decisions and actual implementation. The API should even have some partial functionality coming up very soon. Stay tuned.