Mixing in MongoDB

I have started using MongoDB for specific use cases in production. I don't see the world in black and white, and neither should you. My goal in writing this post is to note down the rationale and list some gotchas when using it with MongoMapper.

MongoDB provides document-oriented storage, with replication, indexing and rich query support. It blends aspects of distributed key-value stores by keeping most of the data in memory, pulls in fine-grained indexing and query language reminiscent of relational databases, provides map-reduce support for data processing similar to Big Table, while optimizing for fast in-place modifications. In the near term, it will provide horizontal scaling and sharding.

The tradeoff for MongoDB is in lack of transactions and being not fully ACID.  This means MongoDB does not offer single-server durability. Data is only eventually consistent. Due to filesystem operational choices (fsync vs. append-only writes, commit log, etc.) MongoDB can lose data during hard server loss.

Realizing all theses aspects, why did I choose to go with MongoDB?

Read the rest of this post »

Twilio on Rails

A few weeks ago, I looked up Twilio in conjunction with a telephony project. I was intrigued by the promise of a simple REST web API for building SMS and voice based interactions within web apps. In prototyping, I ended up developing a Rails plugin called Twilioflow which allows simplified development and testing of telephony apps.

Having spent some time at AT&T/Lucent Bell Labs, I had heard my share of war stories from the programmers and managers who worked on the venerable Audix system. From that perspective, Twilio is yet another example of creative destruction made possible by improvements in the general purpose computing platform coupled with an extensible network protocol (HTTP). Operationally, Twilio goes even further by harnessing Amazon's EC2 infrastructure and we can be sure they have numerous other tricks up their sleeve.

Read the rest of this post »

AirDB can has_many_through

We now have a better, declarative and richer mechanism of specifying model relationships. In the process, AirDB now has support for has_many_through associations. As discussed earlier for join table attributes, the cleanest way to express the association is a has_many_through, where two models have a many to many relationship through an intermediate model. Join table attributes are a nice hack for someone who does not want to rewrite their application code to introduce the intermediate Model class. In some cases, such as self-referential many-many associations, or when you are in the early stages of designing the schema and the application, you may find it appealing to manage the join table as a first-class Model. If so, then has_many_through will help you.

Read the rest of this post »

UI-driven Database Snapshot & Restore

For most people, backing application databases is usually (touch wood) a one-way operation. You take your precautionary measures, employ some kind of hosted or scripted backup solution, and hope that you only infrequently, if ever, have to restore from an old database.

But, what if you want to take a database snapshot at any time, keep a list of such snapshots and restore at will from any of them at any time? And you want to do it from the UI? Sounds far-fetched?

Here is one scenario.

  1. Your sales guy has lined up multiple demos for your app and he wants to tailor the data and demo aspects for each client.
  2. Your dev team is too busy to spend time crafting a whole new admin console just to make a whiz-bang demo.
  3. ssh? mysqldump? redirect output? >, < #@*?

With Rails and Rake, here is how you can make your sales guy happy and get him to buy you some Mai Tais.

Read the rest of this post »

Rails-style jQuery UI Menu

Some people love the drop-down, drill-down fly-out menu, which is now working its way into the official jQuery UI code-base.

The menu looks great, but is a bit tedious to construct. In fact, its downright onerous for the railszers used as we are to select_tag and options_for_select.

Ruby to the rescue!

The rails-jquery-dropdown helper generates the drop-down HTML markup, places menu items and values and optionally sub-menus using a passed-in Array, and wires up the Javascript behavior using jQuery.

Get it from GitHub: http://github.com/dkeskar/rails-jquery-dropdown

AirDB Join Table Attributes

In some cases, it becomes necessary to have additional attributes associated with a many-many relationship. This is typically true of "has_and_belongs_to_many" associations which end up mirroring some kind of "membership" between the joined models. For example, a library system might track borrowing of books by patrons, with extra attributes for return dates and accrued fines. The Django guys have a handy example about musicians and bands with extra attributes such as the membership role and date of joining. In my case, I needed to track sharing status of Photos across Photosets. AirDB now has support for such join table attributes. Its a bit of a hack, in the interests of time, code size and abstraction concessions. The new things, which make it all possible include:
  1. An optional argument to Migrator.joinTable()
  2. DB.execute(sql) to allow tweaking an existing join table.
  3. Associator methods: setAttr, findAllByAttr, countByAttr, getAttrVal.
Here are some actual code excerpts (in the Photoset model)

Read the rest of this post »

Cuing FlixIQ

Give the Clue, Flix I Cue. That's FlixIQ. And it is now live at http://flixiq.com If you have not yet clicked to see for yourselves, here is some wordiness.

FlixIQ lets you cue a video clue to create and share quizzes. We are talking video clues here, which means movies, music, sports, or anything where images flickering 24 times a second (a.k.a flix) trigger some memory or mental connection. So, show off your total recall by answering those questions. If that's too easy for you, stump others by finding a telling scene, and create tough challenges for others. Verdomme, dat is kicken.

Harmonious Hashing

Working in ActionScript and JavaScript, one gets used to getting and setting known Object properties using the dot-accessor. 
var h = {}
h.foo = "bar";
For the lazy and the brave, the same can be accomplished in Ruby. 
class Hash
  def method_missing(m, *args)
    if args.size > 0 
      self[m] = args.first
    elsif self.include?(m)
      self[m] 
    elsif self.include?(m.to_s)
      self[m.to_s]
    end  
  end
end
The newly assigned key is a Symbol, a desirable default.

AirDB Updates

A summary of the several improvements committed to AirDB in the last five weeks. 
  1. Support for has-many and belongs-to associations
  2. Support for multiple associations for a given Modeler class.
  3. More smarts and existence checks for automatic schema migration
  4. Call-chaining for Associations e.g. post.author.comments.count
  5. Miscellaneous fixes, improvements and optimizations
Highlights of the major new additions since I wrote about the initial design.  The Migrator handles schema versioning, checking and automatic migrations. 
public class Migrator implements IMigratable
{
  public function Migrator(klass:Class, options:Object, directives:Array) {
     // setup table structures, prototype fields, existing schema
     DB.migrate(this);
  }
  // Supported migration directives

  // create a table after processing the provided function block
  public function createTable(block:Function):void { ..}

  // specify a column, if it does not exist, it is created.
  public function column(name:String, dataType:uint, options:Object = null):void {..}

  // make a join table to handle many-many associations with specified class
  public function joinTable(klass:Class):void { .. }

  // map a foreign key for belongs-to association
  public function belongsTo(klass:Class):void { .. }
}
The Associator handles associations which are specified via class meta-data. The improvements under the hood make it easier to make associations between Modeler object instances. For example 
// Use the record ID of the target to specify associations
// Post has-many comments
post.comments.push(3);

// Chain the target for further calls
// Post belongs-to author and Author has-many comments

// Obtain field value of associated target
var nm:String = post.author.name;

// Make further associations on associated target
post.author.comments.push(new Comment({title: 'hello there'}));
Join queries and foreign-key lookups are now a breeze with AirDB.

AirDB available at github

I received a request today for a look at the code in response to my post on Designing an ActionScript ORM for AIR.  So, I took a few hours to take the ActionScript ORM code, add a working example and put it out on github.  AirDB is now available here:  http://github.com/dkeskar/airdb/tree/master Suggestions, improvements and contributions are welcome and appreciated.