Tomcat and MongoDB

When I started with MongoDB I wanted to use it with everybody's old favourite application server, Tomcat.
There is no point reinventing the wheel so I hunted for usage examples.  I struggled to find any best practice implementation, so I decided to hack my own together.

A requirement I had was I had was I wanted to be able to deploy the web app to a range of different environments, so I needed the service to be flexible.  The simplest way I could think of was pulling the config from the application's web.xml file.


The Mongo object instance represents a pool of connections to the database so you will only need one object of class Mongo even with multiple threads.

web.xml snippet:


MongoService snippet:


Then from the data access object you can specify and authenticate to the relevant DB.

XxxDAO snippet:


This has been working like a dream for me proving to be an extremely convenient way to easily connect to Mongo.

Convert HTML form to JSON and POST using jQuery

Reinventing the wheel is a pet hate of mine; I could list quite a few actually, but that's not the point of this post.  Sometimes it takes me so long to look for a wheel that I think has or should have already been invented that it would have been easier just to invent the wheel myself.

Recently I was playing around with jquery, pulling fields from a form and posting them as a JSON object to a set of services.  Nothing fancy, nothing difficult there.  However, when the data model started to mature beyond the most basic structure I soon realised I need an easy, repeatable way to pull structured/nested data from a a form and convert it into a JSON object prior to posting.

To create the following structure:


You will need to create the following references:


This is a complete working example that will parse the form, create the JSON obbject, POST it to a service and display the returned object in a table.



The form above will produce the following JSON.

Jersey Annotation

Is your application server is throwing up the following warning at startup?

WARNING: A sub-resource method, public java.lang.String PATH_TO_METHOD, with URI template, "/", is treated as a resource method

Then the most likely cause of is that you have already defined a @Path("/blah") for the class. There is no need to specify @Path("/"), using @Get at the method level is enough to tell Jersey that it is default method for entire class.

Testing your document structure for inconsistencies within MongoDB

One of the advantages with schema-less design is that it works well for prototyping; you can have a collection of documents with each of the documents of variable structure. You can modify the document structure for one, some or all documents within the collection all without requiring a schema for the collection or each and every document.

However, this is also a disadvantage during prototyping; there are no constraints to stop documents within the same collection having variable structure. Deliberate updates to a document succeed silently as do accidental updates; ie when you update a document with a subdocument hanging off the wrong node. So when you assume you have consistency across all the documents, within a collection, but don't, you will run into some issues. You could also argue here that you're not coding defensively enough if you're not checking consistency at the time of execution; I'm not going to go into that right now though.

That exact structure inconsistency happened to me, and I ended up going down a rabbit hole. The smart thing to do was to blat the DB and recreate it during each test, but there were reasons that I didn't do that, which again I'm not going to go into here. Additionally the error that was coming back from performing an operation on the inconsistent structure wasn't obvious and didn't indicate to me that there was document structure inconsistencies, but that's another story.

Anyhoo, I didn't want this to happen again, so to verify the structural consistency of a collection I now pull in a json template; an example of the structure of the document I'm expecting to find within the collection I'm working with, and compare it to the collection in the DB. You can define your template in a json/txt file or you can manually create the DBObject. Simply put, I perform a symmetrical diff on the documents that are contained in the collection(s) I'm working with, and report on any additional field not defined in the template, I also report on any field that is defined in the template but not the document.

This example creates a DBObject with a few NVPs at the root, a couple as subdocument NVPs and finally an array.

We use this 'template' to compare against the documents within the tests collection.

This is all very lightweight but as a method to verify crude consistency it is very handy, for me anyway.

Security and MongoDB

The MongoDB Security Model has some scope for improvement.

1. As default, Authentication is off. MongoDB is designed to run in a 'trusted' environment, depending on the network configuration to ensure the security of the environment.
2. Pre v2 authentication is not supported within a sharded deployment.
3. Once authenticated a user has full read-write to the entire DB. There is no concept of roles or groups or such like.

Now, I wanted to deploy with Security turned up to the max, so here I will present a practical example.

This is a typical set-up I'd use for my development environment; separate machines each running mongod.

Firstly, let's set up the replicaset.
On the primary we can define and run the replicaset config:

Nothing fancy here, all standard stuff. After a few minutes the dust settles and the primary and secondary identify themselves. You can check the status of the replicaset with:

Now, on each box in the replicaset you'll need to create a key (must be valid Base64 and 1K or less):

You can now bring down both instances. Either hit ctrl+c or do it the right way:

Now we can start each of the mongod instances with the keyFile:

Word of warning here. I spent a significant amount of time trying to set this up on v2.0.0 - yes I know it is a bit dumb to go with a x.0.0 version, but you'd think that something as basic/fundamental as this would be thoroughly tested. Well, suffice to say I ended up moving to the latest binary to get this basic functionality to work. It was annoying at the time, but it certainly made me read the available documentation multiple times.
So now on the primary we can create the admin user aka root, super...

As we have a replicaset these users will existing in admin and mydb on both instances.

This whole process should take about 5 minutes to configure provided you're using a stable, well tested version.

I got unlucky and wasted hours on a buggy version grappling with errors that I couldn't decipher, feeling a bit denvercoder9 (http://xkcd.com/979/).

I went through the pain so you don't have to...

A simple guide to finding distinct array values in a MongoDB collection

So you want to find unique values within an array within a document in a collection? A reasonable request.

In ANSI SQL you'll be using DISTINCT, JOINS and GROUP BYs, stuff you're used to, but in the NoSQL realm your best bet is mapreduce.

It might seem a little bit like hard work, and probably a little intimidating at first, but it is certainly worth it; mapreduce is an extremely powerful tool.

Set up the collection, the map and reduce functions, and execute the mapreduce command:

But now you want to find unique values within an array within each document in an entire collection.

The map function in this example iterates over each of the items and emits the key/value of each array element.

The reduce function aggregates the key/value from each of the emits from the map function. In this example we're looking at unique keys and maintaining a count of the unique keys.

If you're looking to find the distinct array elements for a single document, simply specify the document index. For the entire collection, just leave the query out. *simples*

Simple mongodb equivalents

After making a noob error today, I realised that some of the documentation could use a boost.

By default mongo returns the entire document.

All fields for the matched document will be returned

And in java



To return only certain fields, you need to let mongo know which ones you want


The will only return the fields meh and feh from the matched document

And in Java


You can also say 'all fields except for...'



This will return all the fields in the matched document expect for meh

And in Java