After a review of Chililog over the holiday period, I think the next steps is to make Chililog more scalable.

Firstly, I think we need to replace HornetQ with Flume. Flume allows us to funnel and aggregate log events. It also has agents to collect logs so that we don’t have to write our own.

Also, I think we need to replace mongoDB with the likes of Cassandra/DymanoDB. From my understanding, mongoDB is great if you have as much RAM as your data + indexes due to its use of memory mapped files. As such, storing a lot of log files is not a good fit.

Lastly, I need to upgrade the UI from Sproutcore V2 to EmberJS.

However, Flume has just release version 1.0.0. EmberJS Data is still at alpha and without the nested store functionality that I need. Cassandara and DynammoDb are also freshly minted. Because of this, I am going to let the dust settle for a few months before starting Chililog V2.

In the mean time, I am going to get started with the monitoring side of Chililog.