JMeter multiline tooltip hack

I found something a bit annoying with mongometer; it doesn't handle tooltips very well.

Let me explain.

I wanted to show the full text for each of the settings for MongoOptions.

Take the autoConnectRetry.shortDescription property for example.

autoConnectRetry.shortDescription=If true, the driver will keep trying to connect to the same server in case that the socket cannot be established. There is maximum amount of time to keep retrying, which is 15s by default. This can be useful to avoid some exceptions being thrown when a server is down temporarily by blocking the operations. It also can be useful to smooth the transition to a new master (so that a new master is elected within the retry time). Note that when using this flag: - for a replica set, the driver will trying to connect to the old master for that time, instead of failing over to the new one right away - this does not prevent exception from being thrown in read/write operations on the socket, which must be handled by application Even if this flag is false, the driver already has mechanisms to automatically recreate broken connections and retry the read operations. Default is false.

Now this may be seen as being overly verbose. Fair enough, but I wanted to show all of this text so that users don't keep having to refer to the API docs.

The problem is, because there is so much text on a single line, it extends off both ends of the screen and causes unwanted side effects when multiple virtual desktops are in use.



No problem I thought, as I cast my mind back to my Swing development days. I'll just wrap the text in html tags and create a mutliline tooltip.

autoConnectRetry.shortDescription=<html><b>autoConnectRetry</b><br><br>If true, the driver will keep trying to connect to the same server in case that the socket cannot be established.<br>There is maximum amount of time to keep retrying, which is 15s by default.<br>This can be useful to avoid some exceptions being thrown when a server is down temporarily by blocking the operations.<br>It also can be useful to smooth the transition to a new master (so that a new master is elected within the retry <br>Note that when using this flag: - for a replica set, the driver will trying to connect to the old master for that time, instead of failing over to the new one right away - this does not prevent exception from being thrown in read/write operations on the socket, which must be handled by application Even if this flag is false, the driver already has mechanisms to automatically recreate broken connections and retry the read <operations. <br><br>Default is false.</html>



Oh dear. It seems it isn't going to be that easy. The tooltip now simple displays the html tags.

Time to don my deerstalker and get out the magnifying glass. Let's work backwards.

I'm extending BeanInfoSupport and as such I'm depending on GenericTestBeanCustomizer to render the GUI. Let's look to see what is happening during the rendering process.

core\org\apache\jmeter\testbeans\gui\GenericTestBeanCustomizer.java(597)

text = propertyToolTipMessage.format(new Object[] { desc.getName(), desc.getShortDescription() });

OK. So we're passing the name and short description to the MessageFormat instance.

If we scroll up a bit we'll see the pattern that is being applied to this MessageFormat.

core\org\apache\jmeter\testbeans\gui\GenericTestBeanCustomizer.java(283)

propertyToolTipMessage = new MessageFormat(JMeterUtils.getResString("property_tool_tip")); //$NON-NLS-1$

Not quite. We have a little more work to do here. JMeterUtils is being used to fetch resources. Let's have a quick peek in there.

core\org\apache\jmeter\util\JMeterUtils.java(371)

ResourceBundle resBund = ResourceBundle.getBundle("org.apache.jmeter.resources.messages", loc); // $NON-NLS-1$

OK. We're finally there.

core\org\apache\jmeter\resources\messages.properties(696)

property_tool_tip={0}\: {1}

Let's update this to handle html. I don't want it to break anything else, so I'm going to apply the minimum I can get away with.

core\org\apache\jmeter\resources\messages.properties(696)

property_tool_tip=<html><b>{0}</b><br><br>{1}</html>

OK. I told a teenie weenie lie; I didn't need to add the bold tag and the two new lines, but I couldn't help it. Now the properties file looks like this:

autoConnectRetry.shortDescription=If true, the driver will keep trying to connect to the same server in case that the socket cannot be established.<br>There is maximum amount of time to keep retrying, which is 15s by default.<br>This can be useful to avoid some exceptions being thrown when a server is down temporarily by blocking the operations.<br>It also can be useful to smooth the transition to a new master (so that a new master is elected within the retry <br>Note that when using this flag: - for a replica set, the driver will trying to connect to the old master for that time, instead of failing over to the new one right away - this does not prevent exception from being thrown in read/write operations on the socket, which must be handled by application Even if this flag is false, the driver already has mechanisms to automatically recreate broken connections and retry the read <operations. <br><br>Default is false.

I've removed the opening and closing html tags, along with the bolding of the property name and the two new lines, as they are all now present in the format pattern which will be applied to all tooltips across the application. All we need to do when we want any markup within tooltips is simply add it to the property value and it will automagically be rendered. If you don't want markup, then don't add any.



This looks so much better and has the added benefit of addressing the unwanted side effects when using multiple virtual desktops.

I'd imagine this isn't the best way to implement this, but maybe it is; I didn't spend much time on this and I wouldn't be surprised if there is a well documented/clever-er way of achieving the same result without having to hack a global message format pattern.

If you don't like the bold and new lines, then the change to the pattern should be as follows:

core\org\apache\jmeter\resources\messages.properties(696)

property_tool_tip=<html>{0}\: {1}</html>

SpiderMonkey to V8, mongometer and the Aggregation Framework

I previously posted a comparison that covered running some simple queries against versions 2.2.2 and 2.3.2 of MongoDB. They were pretty basic examples, I just wanted to demonstrate one of the uses of mongometer; comparing the relative performance of MongoDB releases and the MongoDB scripts you write to run on them.

Now I'm going to knock the complexity up a notch and perform a comparison between the different releases of MongoDB, their underlying JavaScript engines using another relatively new feature (added in version 2.1), the Aggregation Framework. I'm going to use the example they have in the documentation, mainly so I don't have to make something up.

{
title : "this is my title" ,
author : "bob" ,
posted : new Date () ,
pageViews : 5 ,
tags : [ "fun" , "good" , "fun" ] ,
comments : [
{ author :"joe" , text : "this is cool" } ,
{ author :"sam" , text : "this is bad" }
],
other : { foo : 5 }
}

db.articles.aggregate(
{ $project : {
author : 1,
tags : 1,
} },
{ $unwind : "$tags" },
{ $group : {
_id : { tags : "$tags" },
authors : { $addToSet : "$author" }
} }
);

I'm populating the collection with data in the form as described above. The only thing I'm adding is that I'm using the JMeter CSV Data Set to populate the author attribute.

Version 2.2.2
So let's make sure we're starting from a clean slate.

$ /usr/lib/mongodb/2.2.2/bin/mongod --port 27000 --dbpath /data/db/2.2.2 --logpath /data/db/2.2.2/mongod.log

$ ps -ef | grep mongo
4974 /usr/lib/mongodb/2.2.2/bin/mongod --port 27000 --dbpath /data/db/2.2.2 --logpath /data/db/2.2.2/mongod.log

$ ./mongo --port 27000

> show dbs
local 0.078125GB
test (empty)



> show dbs
aggregation 0.203125GB
local 0.078125GB
test (empty)

> use aggregation
switched to db aggregation
> db.dropDatabase()
{ "dropped" : "aggregation", "ok" : 1 }

$ sudo kill -15 4974

Version 2.3.2
Let's ensure we have that same clean slate as with Version 2.2.2

$ /usr/lib/mongodb/2.3.2/bin/mongod --port 27001 --dbpath /data/db/2.3.2 --logpath /data/db/2.3.2/mongod.log

$ ps -ef | grep mongo
1463 /usr/lib/mongodb/2.3.2/bin/mongod --port 27001 --dbpath /data/db/2.3.2 --logpath /data/db/2.3.2/mongod.log

$ ./mongo --port 27001

> show dbs
local 0.078125GB
test (empty)



> show dbs
aggregation 0.203125GB
local 0.078125GB
test (empty)

> use aggregation
switched to db aggregation
> db.dropDatabase()
{ "dropped" : "aggregation", "ok" : 1 }

$ sudo kill -15 1463

Conclusions
I ran this a few times and the results were consistent. I'll knock it up another notch over time, and hopefully draw out some useful conclusions, until then, you can draw your own.

Suggestions and comments welcome.

SpiderMonkey to V8 and mongometer

With 10gen switching the default JavaScript engine for MongoDB 2.3/2.4 from SpiderMonkey to V8 I thought I'd take the opportunity to compare the relative performances of the releases using mongometer. Being a Security bod, I really should have looked at the Additional Authentication Features first... Hey ho.

I'll document the steps taken during the comparison, including the set up, so this can be repeated and validated - just in case anyone is interested - but mainly so I can remind myself of what I did; memory, sieve.

The set up
I'm going to install 2.2.2 and 2.3.2 side-by-side on a dedicated machine. I'll then use the latest version of the Java driver with mongometer.

$ wget http://fastdl.mongodb.org/linux/mongodb-linux-x86_64-2.3.2.tgz
$ wget http://fastdl.mongodb.org/linux/mongodb-linux-x86_64-2.3.2.tgz.md5

I got a 403 response for this request...

$ wget http://fastdl.mongodb.org/linux/mongodb-linux-x86_64-2.2.2.tgz
$ wget http://fastdl.mongodb.org/linux/mongodb-linux-x86_64-2.2.2.tgz.md5

$ md5sum -c mongodb-linux-x86_64-2.2.2.tgz.md5
md5sum: mongodb-linux-x86_64-2.2.2.tgz.md5: no properly formatted MD5 checksum lines found

Grrr. An md5 file is supposed to be the checksum (then x2 spaces) and then the filename of the file being checksummed. I'll have to eyeball them instead, well, eyeball the one that I could actually download...

$ md5sum mongodb-linux-x86_64-2.2.2.tgz
be0f5969b0ca23a0a383e4ca2ce50a39 mongodb-linux-x86_64-2.2.2.tgz
$ cat mongodb-linux-x86_64-2.2.2.tgz.md5
be0f5969b0ca23a0a383e4ca2ce50a39

Configure
$ tar -zxvf ~/mongodb-linux-x86_64-2.2.2.tgz
$ sudo mkdir -p /usr/lib/mongodb/2.2.2
$ sudo mv mongodb-linux-x86_64-2.2.2/* /usr/lib/mongodb/2.2.2/
$ rm -r mongodb-linux-x86_64-2.2.2
$ sudo mkdir -p /data/db/2.2.2
$ sudo chown `id -un` /data/db/2.2.2
$ /usr/lib/mongodb/2.2.2/bin/mongod --port 27000 --dbpath /data/db/2.2.2 --logpath /data/db/2.2.2/mongod.log

$ tar -zxvf ~/mongodb-linux-x86_64-2.3.2.tgz
$ sudo mkdir -p /usr/lib/mongodb/2.3.2
$ sudo mv mongodb-linux-x86_64-2.3.2/* /usr/lib/mongodb/2.3.2/
$ rm -r mongodb-linux-x86_64-2.3.2
$ sudo mkdir -p /data/db/2.3.2
$ sudo chown `id -un` /data/db/2.3.2
$ /usr/lib/mongodb/2.3.2/bin/mongod --port 27001 --dbpath /data/db/2.3.2 --logpath /data/db/2.3.2/mongod.log

Let's check they are running.

$ ps -ef | grep mongod
1795 /usr/lib/mongodb/2.2.2/bin/mongod --port 27000 --dbpath /data/db/2.2.2 --logpath /data/db/2.2.2/mongod.log
2059 /usr/lib/mongodb/2.3.2/bin/mongod --port 27001 --dbpath /data/db/2.3.2 --logpath /data/db/2.3.2/mongod.log

Now, let's kill one (gracefully) and move on to the interesting stuff.

$ sudo kill -15 2059
$ ps -ef | grep mongod
1795 /usr/lib/mongodb/2.2.2/bin/mongod --port 27000 --dbpath /data/db/2.2.2 --logpath /data/db/2.2.2/mongod.log

Now I'm jumping on to another box.

$ wget https://github.com/downloads/mongodb/mongo-java-driver/mongo-2.10.1.jar
$ cp mongo-2.10.1.jar /usr/lib/jmeter/2.8/lib/ext
$ cp ~/IdeaProjects/mongometer/out/artifacts/mongometer_jar/mongometer.jar /usr/lib/jmeter/2.8/lib/ext
$ /usr/lib/jmeter/2.8/bin/jmeter.sh

The tests
The tests are really rather basic; I'll perform an insert into two different databases, and perform finds against those databases.

Version 2.2.2

> show dbs
local 0.078125GB













> show dbs
jmeter 0.203125GB
jmeter2 0.203125GB
local 0.078125GB

> use jmeter
> db.jmeter.find().count()
1000
> db.dropDatabase()

> use jmeter2
> db.jmeter.find().count()
1000
> db.dropDatabase()

$ ps -ef | grep mongo
2690 /usr/lib/mongodb/2.2.2/bin/mongod --port 27000 --dbpath /data/db/2.2.2 --logpath /data/db/2.2.2/mongod.log

$ sudo kill -15 2690
$ ps -ef | grep mongo

Nothing. Let's get the 2.3.2 instance up and running.

$ /usr/lib/mongodb/2.3.2/bin/mongod --port 27001 --dbpath /data/db/2.3.2 --logpath /data/db/2.3.2/mongod.log

$ ps -ef | grep mongo 2947 /usr/lib/mongodb/2.3.2/bin/mongod --port 27001 --dbpath /data/db/2.3.2 --logpath /data/db/2.3.2/mongod.log

Version 2.3.2

> show dbs
local 0.078125GB













> show dbs
jmeter 0.203125GB
jmeter2 0.203125GB
local 0.078125GB

> use jmeter
> db.jmeter.find().count()
1000
> db.dropDatabase()

> use jmeter2
> db.jmeter.find().count()
1000
> db.dropDatabase()

Conclusions
I guess you should draw your own. I ran this a couple of times and am considering scripting it so the environments are cleaned down prior to each run, I could probably add more complex queries too. Perhaps if I find some time next weekend then I will.

If you have any suggestions, please leave a comment.

mongometer v2.0

A while back I knocked up mongometer to compare the relative performance of MongoDB scripts. I then made some minor changes, and since then - and only recently - made additional changes based on feedback.

I've now made slightly more significant changes. I'll now cover them briefly.

The minimum JMeter version is now 2.8, this is because it is dependent on a change introduce in JMeter 2.8.

I've created a MongoSourceElement which is accessible from the context menu; Add -> Config Element -> MongoDB Source Config

This pulls all the MongoDB connection details up a level and allows you to share it between multiple MongoScriptSamplers. This means you only need to define the connection once and associate a source name with the instance. When you create a MongoScriptSampler you can then reference the source name as defined in MongoSourceElement.

MongoScriptSampler now only includes the fields that you need. It allows you to specify the mongo source, the database, the database credentials and of course the script to run.

The way I'm currently using it is to create a MongoSourceElement.
Under that I create a ThreadGroup.
And under that I add the MongoDB Script v2.0, View Results Tree, and Graph Results.

If I have multiple script to run, I use the same grouping of, MongoDB Script v2.0, View Results Tree, and Graph Results. See the image below for an example.



There is the added MongoUtils class which has a static method which allows you to specify multiple hosts (if you want) and breaks it out into each host and port, returning an ServerAddress, which to be honest, I'm surprised isn't available under the Java MongoDB driver.

The other semi-useful addition is the QuickEnvironment. It basically allows you to start up mongod, mongo, jmeter and a tail of the jmeter log. You could do this in a script, but hey ho, I did it in a helper class.

The latest version is available on github.

Please have a play, try and break it, make suggestions, give me feedback.

JMeter Bug Bean

I came across a potential bug in JMeter when adding support for MongoOptions within mongometer. It's not a big deal in itself as I worked around it, but it was slightly frustrating at the time.

Implementing the MongoOptions seemed rather innocuous at the time; after all it's just another bunch of properties the user can alter. I ended up just adding them all at once. Imagine my disappointment when the extension failed to load in Jmeter as intended. To keep a long story short, the offender was wTimeout. Now, wtimeout (is one of a group a rather poorly named variables, others include j and w ), but why was there an issue with wtimeout?

The error of my ways

This is the error that was generated when starting up JMeter:


This is how it was defined, firstly in ScriptSamplerBeanInfo.java


Next, in ScriptSamplerResource.properties


And finally, in ScriptSampler.java


I checked and double checked the spelling. It didn't really make sense, it should've been fine. Out of all the properties that were defined, the wTimeout property was the odd one out, in that it was the only one that had the pattern of lowercase{1}uppercase{N}. All the others were either lowercase{1}, lowercase{N} or lowercase{N}uppercase{N}. So long story, short; switching from wTimeout to doubleuTimeout resolved the issue.


Next, in ScriptSamplerResource.properties


And finally, in ScriptSampler.java


It seems that when performing the creation of the property, if there is only a single lowercase character followed by an uppercase character, we get an error.

The moral of the story

Hmmmm. I don't really do morals, but if this is a bug, then it needs to be fixed. Either way j, w and wtimeout are really rather poor names for variables.

Creating your project with IntelliJ

Now we have a development environment configured, lets create the project and start taking it to the next level.

Creating a project in IntelliJ

New Project
New Project from Scratch
Enter the name: mongometer
Select Maven Module
Next
Select Create from archetype (I chose maven j2ee simple)

Arrrrg! Don't do this! Man, IntelliJ sat spinning the tyres for an age.  I ended up killing it and starting again.

To delete a project, close intellij and remove the project folder
projects are in IdeaProjects
$rm -r ~/IdeaProjects/mongometer

Second time around
New Project
New Project from Scratch
Enter the name: mongometer
Select Maven Module
Next
Deselect the create from archetype

Create a package
Create the classes and the properties file

Fix imports
Add dependencies to the pom.xml

This is what I need for mongometer

• junit4.10
  
• mongodb 2.7.2
  
• ApacheJMeter_core 2.6
  
• jorphan 2.6
  

Push to GitHub

$cd ~/IdeaProjects/mongometer
$git init
$git add *.java
$git add *.properties
$git add *.xml
$git commit -m 'Initial upload of the project'
$git remote add origin ssh://github.com/yourname/mongometer.git
$git push -u origin +master

Voilà! You can go to your repo, and the project files are there.  Happy days.

Points of interest

• The + performs a force push to github.  I'm guessing this happened because when I created a new repository on GitHub I opted to create the README file.  I won't do this again.
  
• IntelliJ complained about setting up the repo outside of IntelliJ.  No problem, you just add the repo inside IntelliJ.  I've been an Eclipse person for years (though we've decided that we'd like spend some time apart) so I'm not 100% sure of how to do this inside IntelliJ just yet.
  
• This is a great site for listing dependencies http://mvnrepository.com/ for setting up your pom.xml

Configuring the mongometer Development Environment

As I've already set up a git repository on Ubuntu and published the step-by-step, command-by-command instructions for doing this, I thought it would be worth publishing the same guides for setting up the entire development environment on Ubuntu.

Set up your JDK

Download
http://www.oracle.com/technetwork/java/javase/downloads/jdk-7u4-downloads-1591156.html

Configure
$tar -xvf ~/Downloads/jdk-7u4-linux-x64.tar.gz
$sudo mkdir -p /usr/lib/jvm/jdk1.7.0
$sudo mv jdk1.7.0_04/* /usr/lib/jvm/jdk1.7.0/
$rm -r jdk1.7.0_04
$sudo update-alternatives --install "/usr/bin/java" "java" "/usr/lib/jvm/jdk1.7.0/bin/java" 1
$sudo update-alternatives --install "/usr/bin/javac" "javac" "/usr/lib/jvm/jdk1.7.0/bin/javac" 1
$sudo update-alternatives --install "/usr/bin/javaws" "javaws" "/usr/lib/jvm/jdk1.7.0/bin/javaws" 1
$mkdir ~/.mozilla/plugins/
$ln -s /usr/lib/jvm/jdk1.7.0/jre/lib/amd64/libnpjp2.so ~/.mozilla/plugins/

Set up MongoDB

Download
http://www.mongodb.org/dr/fastdl.mongodb.org/linux/mongodb-linux-x86_64-2.0.6.tgz/download
$md5sum ~/Downloads/mongodb-linux-x86_64-2.0.6.tgz
69eece640fcb1684190a4585f31df954

Configure
$tar -zxvf ~/Downloads/mongodb-linux-x86_64-2.0.6.tgz
$sudo mkdir -p /usr/lib/mongodb/2.0.6
$sudo mv mongodb-linux-x86_64-2.0.6/* /usr/lib/mongodb/2.0.6/
$rm -r mongodb-linux-x86_64-2.0.6
$sudo mkdir -p /data/db
$sudo chown `id -un` /data/db
$/usr/lib/mongodb/2.0.6/bin/mongod --dbpath /data/db --logpath /data/db/mongod.log
$mongod --config /etc/mongod.conf

(from a new terminal start the shell)
$cd /usr/lib/mongodb/2.0.6/bin/
$./mongo
> db.test.save( { a: 1 } )
> db.test.find()

Set up Maven

Download
http://www.apache.org/dyn/closer.cgi/maven/binaries/apache-maven-3.0.4-bin.tar.gz
$md5sum ~/Downloads/apache-maven-3.0.4-bin.tar.gz
e513740978238cb9e4d482103751f6b7

Configure
$tar -xzvf ~/Downloads/apache-maven-3.0.4-bin.tar.gz
$sudo mkdir -p /usr/local/maven/3.0.4
$sudo mv apache-maven-3.0.4/* /usr/local/maven/3.0.4
$rm -r apache-maven-3.0.4

$sudo gedit /etc/environment

JAVA_HOME="/usr/lib/jvm/jdk1.7.0"
M2_HOME="/usr/local/maven/3.0.4"
MAVEN_HOME="/usr/local/maven/3.0.4"
M2="/usr/local/maven/3.0.4/bin"
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/maven/3.0.4/bin"

Verifiy the configuration
Logout and login

$mvn -version
$mkdir ~/.m2
$sudo chown `id -un` -R ~/.m2

Set up IntelliJ

Download
http://www.jetbrains.com/idea/download/

$tar -xvf ~/Downloads/ideaIC-11.1.2.tar.gz
$cd ~/idea-IC-117.418/bin
$chmod +x idea.sh
$./idea.sh

(this is the initial install of intellij, so take the second option)
File -> Project Structure
Platform Settings -> SDKs
Add -> JSDK
 /usr/lib/jvm/jdk1.7.0/
 Select -> OK

Set up JMeter

Download
http://mirror.lividpenguin.com/pub/apache//jmeter/binaries/apache-jmeter-2.7.tgz
$md5sum ~/Downloads/apache-jmeter-2.7.tgz
73435baa6ed99c528dacfa36c7e1f119

Configure
$tar -zxvf ~/Downloads/apache-jmeter-2.7.tgz
$sudo mkdir -p /usr/lib/jmeter/2.7
$sudo mv apache-jmeter-2.7/* /usr/lib/jmeter/2.7/
$rm -r apache-jmeter-2.7
$/usr/lib/jmeter/2.7/bin/jmeter.sh

Performance testing MongoDB

So, this morning I was hacking around in the mongo shell. I had come up with three different ways to aggregate the data I wanted, but wasn't sure about which one I should subsequently port to code to use within my application.

So how would I decide on which method to implement? Well, lets just choose the one that performs the best. Ok, how do I do that? Hmmm. I could download and install some of the tools out there, or I could just wrap the shell code in a function and add some timings. OR, I could use the same tool that I use to performance test everything else; JMeter. To me it was a no brainer.

So how do we do it?

There is a full tutorial here.

Simply put, you need to do the following:

1. Create a Sampler class. 
2. Create a BeanInfo class. 
3. Create a properties file. 
4. Bundle up into a jar and drop into the apache-jmeter-X.X\lib\ext folder 
5. Update search_paths=../lib/ext/mongodb.jar in jmeter.properties if you place the jar anywhere else. 

How I did it

I tend to have a scratch pad project set up in my IDE, so I decided just to go with that. Just to be on the safe side, I imported all the dependencies from:

• apache-jmeter-X.X\lib 
• apache-jmeter-X.X\lib\ext 
• apache-jmeter-X.X\lib\junit

I then created the two classes and the properties file.



I then exported the jar to apache-jmeter-X.X\lib\ext, and fired up jmeter.

Go through the normal steps to set the test plan up:

1. Right click Test Plan and add a Thread Group. 
2. Right click the Thread Group and add a Sampler, in this case a MongoDB Script Sampler. 
3. Add your script to the textarea; db.YOUR_COLLECTION_NAME.insert({"jan" : "thinks he is great"}) 
4. Run the test

Happy days.  You can then use JMeter as you would for any other sampler.


Future enhancements

This is just a hack that took me 37 minutes to get running, plus 24 minutes if you include this post. This can certainly be extended to allow you to enter the replicaset config details for instance and to pull the creation of the connection out so we're not initiating this each time run a test.