Showing posts with label Mongo. Show all posts
Showing posts with label Mongo. Show all posts

Saturday, 2 February 2013

MongoDB Authentication

I recently updated mongometer to make it a bit more flexible. Shortly after releasing the new version, one of the users fed back an issue via a comment on the post. I booted up my machine, opened up my IDE, found the issue and had pushed the fix out to github within half-an-hour.

This isn't a quick turn-around, success story post. It quickly dawned on me that if I was going to do anything in the future with mongometer, I should really know a little more about how a user authenticates against a database within MongoDB. (I don't want to spend more than an hour or so on this as I've just cracked open a bottle of Nyetimber Classic Cuvee - I'm also cooking a chicken pie (ping me if you want the recipe) and I'd rather be finished this post before I finish the bottle.) Before diving into any documentation that may exist around MongoDB Security, I'll start with a few observations. So in typical man style, let's kick the tyres and then if required, RTFM.

Start up a mongod instance.

$ /usr/lib/mongodb/2.3.2/bin/mongod --port 27001 --fork --dbpath /data/db/2.3.2 --logpath /data/db/2.3.2/mongod.log
$ ./mongo --port 27001


Create an admin user

> use admin
> db.addUser("mongouser","mongopass")
1


Restart mongod

$ sudo kill -15 $(ps -ef | grep mongo | grep -v grep | cut -f8 -d" ")
$ /usr/lib/mongodb/2.3.2/bin/mongod --port 27001 --fork --auth --dbpath /data/db/2.3.2 --logpath /data/db/2.3.2/mongod.log
$ ./mongo --port 27001


Authenticate to admin

> use admin
switched to db admin
> db.aut("mongouser","mongopass")
Thu Jan 31 13:53:31.271 javascript execution failed (shell):1 TypeError: Property 'aut' of object admin is not a function
db.aut("mongouser","mongopass")
^

> db.aut("mongouser","mongopass")


Ooops. Fat-fingered it. Hang on, I think I've found Issue #1

Issue #1
If an admin user mistypes the auth command and not the credentials, then the actual credentials stay in the shell history, which persists across sessions. Any other user could potentially come along and view the shell history and pick the credentials up.

On the other hand, if the command is correct and either the username or password or both are incorrect, or indeed if the authentication attempts succeeds, then the command is not kept in the history. (The command history for the mongo shell is available in the same way as on a linux box - using the up arrow)

> db.auth("mongouser","mongopass0")
{ ok: 0.0, errmsg: "auth fails" }
0
> db.auth("mongouser0","mongopass0")
{ ok: 0.0, errmsg: "auth fails" }
0
> db.auth("mongouser0","mongopass")
{ ok: 0.0, errmsg: "auth fails" }
0


Ok. Let's authenticate against admin and continue.

> use admin
switched to db admin
> db.auth("mongouser","mongopass")
1


Oooops. I almost missed one there.

Issue #2
Until the mongod instance is restarted, any user can...

> use admin
switched to db admin
> db.system.users.find()
{ "_id" : ObjectId("510a58c6de50e136190f9ed7"), "user" : "mongouser", "readOnly" : false, "pwd" : "c49caa1cb6b287ff6b1deaeeb8f4d149" }


...grab the usernames and hashes.

So, now that I've restarted the mongod instance, any user is going to have to authenticate against admin to be able to view the contents of system.users.

Now, continuing on from entering incorrect credentials, I'm going to launch a dictionary attack and see what happens. Oh dear. Found another issue.

Issue #3
There is no lock-out. I wrote a quick hack to connect to the mongod instance, to switch over to admin and attempt to log in. Using a rather large dictionary (with "mongopass" tacked on at the end) I attempted to log in over a million times. This was only a crude single-threaded attempt that took around 17 seconds to complete, but it shows that there is no account lock out. I'm confident I could put together a multi-threaded brute-forcer if required. I'll need to look into this further to see if there is any brute forcing/dictionary attack alerting that can be configured or whether there is a lock-out policy that can be applied. I'm not ready to RTFM just yet.

Let's take a closer look at the format of the password in system.users.

c49caa1cb6b287ff6b1deaeeb8f4d149

That looks like an MD5 to me. Let's take a look in the code, which is available to cruise on github.

Wow! I got luck straight off-the-bat. db.js has the following method:

function _hashPassword(username, password) {
    return hex_md5(username + ":mongo:" + password);
}


With hex_md5 then referencing native_hex_md5 within utils.cpp:

void installGlobalUtils( Scope& scope ) {
    scope.injectNative( "hex_md5" , native_hex_md5 );
    scope.injectNative( "version" , native_version );
    scope.injectNative( "sleep" , native_sleep );
    installBenchmarkSystem( scope );
}

static BSONObj native_hex_md5( const BSONObj& args, void* data ) {
    uassert( 10261, "hex_md5 takes a single string argument -- hex_md5(string)",
    args.nFields() == 1 && args.firstElement().type() == String );
    const char * s = args.firstElement().valuestrsafe();

    md5digest d;
    md5_state_t st;
    md5_init(&st);
    md5_append( &st , (const md5_byte_t*)s , strlen( s ) );
    md5_finish(&st, d);

    return BSON( "" << digestToString( d ) );
}


Time for a quick recap. Just in case you missed anything:
  1. the hashing algorithm is MD5; my least favourite hashing algorithm.
  2. the string to be hashed is in the form username + ":mongo:" + password; using the same "salt" is non-optimal...
  3. the string :mongo: is global; I'm not really sure why it's there at all tbh.
I think this is probably enough to go with for now, else this will turn into a tl;dr and I may exceed my self imposed time constraints.

Thinking back to any discussions I had with regards to MongoDB, the same statements always arose within the context of Security.
  1. Authentication is off by default.
  2. MongoDB was always meant to be deployed in a trusted environment
I have to say that even with authentication on, we still have some gnarly issues. Further, I don't think a trusted environment exists.

Right then, time to RTFM with regards to Security. I'm hoping to find a roadmap defined that will deal with the issues stated above or there are already some mitigating steps that can be taken.

So, there are some Authentication features coming out in the near future. It looks like the new authentication features are only available under the MongoDB Subscriber Edition, I'm not sure what that means tbh... I also came across this know issue, which forms the basis for...

Issue #4
"if a user has the same password in multiple databases, the hash will be the same on all database. A malicious user could exploit this to gain access on a second database use a different users’ credentials." [sic]

Let's break that down.

"if a user has the same password in multiple databases, the hash will be the same on all database."

Yes. Correct. Same username, same password and same "salt" (ie the ":mongo:" string") equals same hash. OK, cool, let's move on.

"A malicious user could exploit this to gain access on a second database use a different users’ credentials." [sic]

A malicious user could exploit this if, and only if they have a non-readonly user on both databases involved.

If they only have readonly access, then they cannot list the system.users collection. In which case they will never see that the hashes are the same across different databases in the first place.

If they are not readonly, then they could list the system.users collection and take the hashed passwords offline to crack.

You're going to have to move into cracking territory if the hashes don't match across databases, in summary:
  1. the user attribute would have be the same. The odds of different users on different databases having the user could be high.
  2. the pwd attribute would have be the same. The odds of different users creating the same pwd is probably quite high.
  3. the "salt" is the same, so it has no real relevance here.
So the problem here is that a user (that is not readonly) can pull all the password hashes for a given database and take them offline to crack. The malicious user already has the user name and the "salt", all they have to find is the password.

Conclusions

Issue #1
This one is a bit of a pain tbh. When the command is entered correctly (ignoring whether the credentials are correct or not) the command is not shown in the history. When the command is not entered correctly, then it is difficult to know what to exclude from the command history. I guess you could retrospectively remove commands that resulted in errors (ie invalid commands) that preceded the authentication. That is not a solution...

Issue #2
There may be an argument that once the admin user is created in system.users in the admin database that a restart should be forced.

Issue #3
A no-brainer. I've written password policies on multiple occasions (what a fun life I live, eh?), account lock-out is password 101.

Issue #4
It seems that creating a "salt" (":mongo:") per database would resolve the issue. Looking at the code, it looks like the implementation is a doddle, a quick and easy win. Adding the option to manually set it would be grand. Implementing a unique "salt" under the covers such that users didn't have to think about it would be equally grand.

So, Nyetimber finished, post finished.

I'm not saying that there is anything in this post that is new or clever, it's a cursory glance. I'm not having a go; everything I've mentioned is merely observation. I install mongo on almost a daily basis because it's a great product, I do however like having a balanced view and identifying any elephants in the room. I'd be interested in any feedback.

Monday, 18 June 2012

Configuring the mongometer Development Environment

As I've already set up a git repository on Ubuntu and published the step-by-step, command-by-command instructions for doing this, I thought it would be worth publishing the same guides for setting up the entire development environment on Ubuntu.

Set up your JDK

Download
http://www.oracle.com/technetwork/java/javase/downloads/jdk-7u4-downloads-1591156.html

Configure
$tar -xvf ~/Downloads/jdk-7u4-linux-x64.tar.gz
$sudo mkdir -p /usr/lib/jvm/jdk1.7.0
$sudo mv jdk1.7.0_04/* /usr/lib/jvm/jdk1.7.0/
$rm -r jdk1.7.0_04
$sudo update-alternatives --install "/usr/bin/java" "java" "/usr/lib/jvm/jdk1.7.0/bin/java" 1
$sudo update-alternatives --install "/usr/bin/javac" "javac" "/usr/lib/jvm/jdk1.7.0/bin/javac" 1
$sudo update-alternatives --install "/usr/bin/javaws" "javaws" "/usr/lib/jvm/jdk1.7.0/bin/javaws" 1
$mkdir ~/.mozilla/plugins/
$ln -s /usr/lib/jvm/jdk1.7.0/jre/lib/amd64/libnpjp2.so ~/.mozilla/plugins/

Set up MongoDB

Download
http://www.mongodb.org/dr/fastdl.mongodb.org/linux/mongodb-linux-x86_64-2.0.6.tgz/download
$md5sum ~/Downloads/mongodb-linux-x86_64-2.0.6.tgz
69eece640fcb1684190a4585f31df954

Configure
$tar -zxvf ~/Downloads/mongodb-linux-x86_64-2.0.6.tgz
$sudo mkdir -p /usr/lib/mongodb/2.0.6
$sudo mv mongodb-linux-x86_64-2.0.6/* /usr/lib/mongodb/2.0.6/
$rm -r mongodb-linux-x86_64-2.0.6
$sudo mkdir -p /data/db
$sudo chown `id -un` /data/db
$/usr/lib/mongodb/2.0.6/bin/mongod --dbpath /data/db --logpath /data/db/mongod.log
$mongod --config /etc/mongod.conf

(from a new terminal start the shell)
$cd /usr/lib/mongodb/2.0.6/bin/
$./mongo
> db.test.save( { a: 1 } )
> db.test.find()

Set up Maven

Download
http://www.apache.org/dyn/closer.cgi/maven/binaries/apache-maven-3.0.4-bin.tar.gz
$md5sum ~/Downloads/apache-maven-3.0.4-bin.tar.gz
e513740978238cb9e4d482103751f6b7

Configure
$tar -xzvf ~/Downloads/apache-maven-3.0.4-bin.tar.gz
$sudo mkdir -p /usr/local/maven/3.0.4
$sudo mv apache-maven-3.0.4/* /usr/local/maven/3.0.4
$rm -r apache-maven-3.0.4

$sudo gedit /etc/environment

JAVA_HOME="/usr/lib/jvm/jdk1.7.0"
M2_HOME="/usr/local/maven/3.0.4"
MAVEN_HOME="/usr/local/maven/3.0.4"
M2="/usr/local/maven/3.0.4/bin"
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/maven/3.0.4/bin"

Verifiy the configuration
Logout and login

$mvn -version
$mkdir ~/.m2
$sudo chown `id -un` -R ~/.m2

Set up IntelliJ

Download
http://www.jetbrains.com/idea/download/

$tar -xvf ~/Downloads/ideaIC-11.1.2.tar.gz
$cd ~/idea-IC-117.418/bin
$chmod +x idea.sh
$./idea.sh

(this is the initial install of intellij, so take the second option)
File -> Project Structure
Platform Settings -> SDKs
Add -> JSDK
 /usr/lib/jvm/jdk1.7.0/
 Select -> OK

Set up JMeter

Download
http://mirror.lividpenguin.com/pub/apache//jmeter/binaries/apache-jmeter-2.7.tgz
$md5sum ~/Downloads/apache-jmeter-2.7.tgz
73435baa6ed99c528dacfa36c7e1f119

Configure
$tar -zxvf ~/Downloads/apache-jmeter-2.7.tgz
$sudo mkdir -p /usr/lib/jmeter/2.7
$sudo mv apache-jmeter-2.7/* /usr/lib/jmeter/2.7/
$rm -r apache-jmeter-2.7
$/usr/lib/jmeter/2.7/bin/jmeter.sh

Thursday, 26 April 2012

Performance testing MongoDB

So, this morning I was hacking around in the mongo shell. I had come up with three different ways to aggregate the data I wanted, but wasn't sure about which one I should subsequently port to code to use within my application.

So how would I decide on which method to implement? Well, lets just choose the one that performs the best. Ok, how do I do that? Hmmm. I could download and install some of the tools out there, or I could just wrap the shell code in a function and add some timings. OR, I could use the same tool that I use to performance test everything else; JMeter. To me it was a no brainer.

So how do we do it?

There is a full tutorial here.

Simply put, you need to do the following:
  1. Create a Sampler class. 
  2. Create a BeanInfo class. 
  3. Create a properties file. 
  4. Bundle up into a jar and drop into the apache-jmeter-X.X\lib\ext folder 
  5. Update search_paths=../lib/ext/mongodb.jar in jmeter.properties if you place the jar anywhere else. 

How I did it

I tend to have a scratch pad project set up in my IDE, so I decided just to go with that. Just to be on the safe side, I imported all the dependencies from:
  • apache-jmeter-X.X\lib 
  • apache-jmeter-X.X\lib\ext 
  • apache-jmeter-X.X\lib\junit
I then created the two classes and the properties file.



I then exported the jar to apache-jmeter-X.X\lib\ext, and fired up jmeter.

Go through the normal steps to set the test plan up:
  1. Right click Test Plan and add a Thread Group. 
  2. Right click the Thread Group and add a Sampler, in this case a MongoDB Script Sampler. 
  3. Add your script to the textarea; db.YOUR_COLLECTION_NAME.insert({"jan" : "thinks he is great"}) 
  4. Run the test

Happy days.  You can then use JMeter as you would for any other sampler.


Future enhancements

This is just a hack that took me 37 minutes to get running, plus 24 minutes if you include this post. This can certainly be extended to allow you to enter the replicaset config details for instance and to pull the creation of the connection out so we're not initiating this each time run a test.

Monday, 12 March 2012

Mongodb MapReduce scope variables

I recently had a requirement for conditional emission from a map function. Essentially I only wanted to emit where the date was within a given range.

In SQL, grouping a count for a given time granularity within a date range would look something like this:

I wasn't 100% sure about the 'right way' to achieve this in NOSql/MongoDB. So this is a solution.
It requires you to know about scope variables. Problem is, I found that scope variables are not very well documented. You can find more about scope variables in the MongoDB documentation MapReduce-Overview. The relevant parts are:

      [, scope : <object where fields go into javascript global scope >]
and
      scope - can pass in variables that can be access from map/reduce/finalize.

Back to this example. First, let's define some data:

Before we implement this, lets get it working at the command-line, in our mongo shell:

What is happening here?

Well, we're selecting the sub-document of this collection where the value of "meh" is "meh". Then we've defined two dates; from and to to represent the boundaries of the date range, we're including these within the MapReduce function call. Basically what this means is that we can use what ever is defined here in the Map function (btw, we can also use them in the Reduce and Finalize functions).

Once we have this working from the shell, it is straight forward to implement it. This is the very same implemented in Java.

Caveat: This is an example of how to use scope variables, I'm sure if you go to any of the Events or gmane.comp.db.mongodb.user group you'll get some advice straight from the 10Gen guys.