Key Stretching; an example

I've had quite a few questions about my previous post (from June 2010) on Passwords since I recently reposted it.

More specifically the questions and comments were around key stretching.

Q.  But doesn't looping that many times slow the password verification step down?
A.  Well yes.  That's kind of the point.  The user may experience negligible latency during the authentication process, this can be tuned to have no or little effect on the user experience.  This same delay is multiplied by the number of brute force attempts made by the attacker.

Q.  The attacker doesn't need the salt or the algorithm if they are going through the front door?
A.  Agreed.  Which is why you'd have some controls and triggers at the front door to alert when abnormal or atypical behaviour is detected.

Q.  If an attack gains access to a system, surely it's game over for user PII?
A.  The thing I like about key stretching is that a hacker needs to have access to the salt, the hashed password and stretching algorithm.  So, if you store the salt in a different table or even DB than the hashed password, the attacker would need to get their mitts on both tables or both DBs. Gaining visibility of the algorithm means that the attacker would also need to have access to, or knowledge about the specific implementation details; ie the source code. Typical environment configurations ensure that there is no way to get to a development environment from a production environment (and vice versa), making it difficult to gain access to the source.

This is a quick and dirty example of what key stretching would look implemented in Java.


This runs in around 0.75 seconds on my local box. Which means every iteration of a dictionary attack or a brute force attack would take the same time.

Caveat: this is purely an example of how you could implement stretching.

mongometer Unhacked

When I first hacked mongometer together, I didn't think that I'd need to spend more than that one hour on both coding and posting on the hack. However, the more I used it, the more I found it useful a way of comparing the relative performance of my scripts, and the more I found it useful, the more I thought that others may find it useful.

(I also found a few shortcomings/annoyances, which is great, because that gives me some incentive to improve the hack.)

1. If I needed to change the hosts, the database, the credentials, you needed to rebuild and redeploy the jar.
   
2. There was no way to tweak the options [http://api.mongodb.org/java/2.7.2/com/mongodb/MongoOptions.html].
   
3. Connections didn't seem to be getting recycled.  You had to shutdown jmeter before the open connections were tried up.
   
4. Wanted to ensure that the connection didn't need to be created every time there was an iteration, and that the connection pooling could be utilised
   

1. Easy enough to the change this, I added the fields that had been frustrating me.

2. Again, easy enough to add these additional properties, though I did come up against a slight frustration

3. This was also slightly frustrating. JMeter didn't seem to be release any of the resources tied up in the Sampler. This was eventually resolved by a bit of experimentation and a bit of 4.

4. This turned out to be a lesson in RTFM. Implementing TestListener allows you to connect to MongoDB at the start of the test run, use the connection pooling during the iterations, and tidy up resources at the end of the run.

Moral
I don't really do 'and the moral of the story is...', but in this case, RTFM is reasonably apt. And, it's quick and easy to hack together something that works, it's not so easy to put together something usable in the longer term; but we all know that.

Request For Comments
If you found this useful, please leave a comment.
If there is a missing feature, please let me know and I'll try and add it.

JMeter Bug Bean

I came across a potential bug in JMeter when adding support for MongoOptions within mongometer. It's not a big deal in itself as I worked around it, but it was slightly frustrating at the time.

Implementing the MongoOptions seemed rather innocuous at the time; after all it's just another bunch of properties the user can alter. I ended up just adding them all at once. Imagine my disappointment when the extension failed to load in Jmeter as intended. To keep a long story short, the offender was wTimeout. Now, wtimeout (is one of a group a rather poorly named variables, others include j and w ), but why was there an issue with wtimeout?

The error of my ways

This is the error that was generated when starting up JMeter:


This is how it was defined, firstly in ScriptSamplerBeanInfo.java


Next, in ScriptSamplerResource.properties


And finally, in ScriptSampler.java


I checked and double checked the spelling. It didn't really make sense, it should've been fine. Out of all the properties that were defined, the wTimeout property was the odd one out, in that it was the only one that had the pattern of lowercase{1}uppercase{N}. All the others were either lowercase{1}, lowercase{N} or lowercase{N}uppercase{N}. So long story, short; switching from wTimeout to doubleuTimeout resolved the issue.


Next, in ScriptSamplerResource.properties


And finally, in ScriptSampler.java


It seems that when performing the creation of the property, if there is only a single lowercase character followed by an uppercase character, we get an error.

The moral of the story

Hmmmm. I don't really do morals, but if this is a bug, then it needs to be fixed. Either way j, w and wtimeout are really rather poor names for variables.

Creating your project with IntelliJ

Now we have a development environment configured, lets create the project and start taking it to the next level.

Creating a project in IntelliJ

New Project
New Project from Scratch
Enter the name: mongometer
Select Maven Module
Next
Select Create from archetype (I chose maven j2ee simple)

Arrrrg! Don't do this! Man, IntelliJ sat spinning the tyres for an age.  I ended up killing it and starting again.

To delete a project, close intellij and remove the project folder
projects are in IdeaProjects
$rm -r ~/IdeaProjects/mongometer

Second time around
New Project
New Project from Scratch
Enter the name: mongometer
Select Maven Module
Next
Deselect the create from archetype

Create a package
Create the classes and the properties file

Fix imports
Add dependencies to the pom.xml

This is what I need for mongometer

• junit4.10
  
• mongodb 2.7.2
  
• ApacheJMeter_core 2.6
  
• jorphan 2.6
  

Push to GitHub

$cd ~/IdeaProjects/mongometer
$git init
$git add *.java
$git add *.properties
$git add *.xml
$git commit -m 'Initial upload of the project'
$git remote add origin ssh://github.com/yourname/mongometer.git
$git push -u origin +master

Voilà! You can go to your repo, and the project files are there.  Happy days.

Points of interest

• The + performs a force push to github.  I'm guessing this happened because when I created a new repository on GitHub I opted to create the README file.  I won't do this again.
  
• IntelliJ complained about setting up the repo outside of IntelliJ.  No problem, you just add the repo inside IntelliJ.  I've been an Eclipse person for years (though we've decided that we'd like spend some time apart) so I'm not 100% sure of how to do this inside IntelliJ just yet.
  
• This is a great site for listing dependencies http://mvnrepository.com/ for setting up your pom.xml

Configuring the mongometer Development Environment

As I've already set up a git repository on Ubuntu and published the step-by-step, command-by-command instructions for doing this, I thought it would be worth publishing the same guides for setting up the entire development environment on Ubuntu.

Set up your JDK

Download
http://www.oracle.com/technetwork/java/javase/downloads/jdk-7u4-downloads-1591156.html

Configure
$tar -xvf ~/Downloads/jdk-7u4-linux-x64.tar.gz
$sudo mkdir -p /usr/lib/jvm/jdk1.7.0
$sudo mv jdk1.7.0_04/* /usr/lib/jvm/jdk1.7.0/
$rm -r jdk1.7.0_04
$sudo update-alternatives --install "/usr/bin/java" "java" "/usr/lib/jvm/jdk1.7.0/bin/java" 1
$sudo update-alternatives --install "/usr/bin/javac" "javac" "/usr/lib/jvm/jdk1.7.0/bin/javac" 1
$sudo update-alternatives --install "/usr/bin/javaws" "javaws" "/usr/lib/jvm/jdk1.7.0/bin/javaws" 1
$mkdir ~/.mozilla/plugins/
$ln -s /usr/lib/jvm/jdk1.7.0/jre/lib/amd64/libnpjp2.so ~/.mozilla/plugins/

Set up MongoDB

Download
http://www.mongodb.org/dr/fastdl.mongodb.org/linux/mongodb-linux-x86_64-2.0.6.tgz/download
$md5sum ~/Downloads/mongodb-linux-x86_64-2.0.6.tgz
69eece640fcb1684190a4585f31df954

Configure
$tar -zxvf ~/Downloads/mongodb-linux-x86_64-2.0.6.tgz
$sudo mkdir -p /usr/lib/mongodb/2.0.6
$sudo mv mongodb-linux-x86_64-2.0.6/* /usr/lib/mongodb/2.0.6/
$rm -r mongodb-linux-x86_64-2.0.6
$sudo mkdir -p /data/db
$sudo chown `id -un` /data/db
$/usr/lib/mongodb/2.0.6/bin/mongod --dbpath /data/db --logpath /data/db/mongod.log
$mongod --config /etc/mongod.conf

(from a new terminal start the shell)
$cd /usr/lib/mongodb/2.0.6/bin/
$./mongo
> db.test.save( { a: 1 } )
> db.test.find()

Set up Maven

Download
http://www.apache.org/dyn/closer.cgi/maven/binaries/apache-maven-3.0.4-bin.tar.gz
$md5sum ~/Downloads/apache-maven-3.0.4-bin.tar.gz
e513740978238cb9e4d482103751f6b7

Configure
$tar -xzvf ~/Downloads/apache-maven-3.0.4-bin.tar.gz
$sudo mkdir -p /usr/local/maven/3.0.4
$sudo mv apache-maven-3.0.4/* /usr/local/maven/3.0.4
$rm -r apache-maven-3.0.4

$sudo gedit /etc/environment

JAVA_HOME="/usr/lib/jvm/jdk1.7.0"
M2_HOME="/usr/local/maven/3.0.4"
MAVEN_HOME="/usr/local/maven/3.0.4"
M2="/usr/local/maven/3.0.4/bin"
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/maven/3.0.4/bin"

Verifiy the configuration
Logout and login

$mvn -version
$mkdir ~/.m2
$sudo chown `id -un` -R ~/.m2

Set up IntelliJ

Download
http://www.jetbrains.com/idea/download/

$tar -xvf ~/Downloads/ideaIC-11.1.2.tar.gz
$cd ~/idea-IC-117.418/bin
$chmod +x idea.sh
$./idea.sh

(this is the initial install of intellij, so take the second option)
File -> Project Structure
Platform Settings -> SDKs
Add -> JSDK
 /usr/lib/jvm/jdk1.7.0/
 Select -> OK

Set up JMeter

Download
http://mirror.lividpenguin.com/pub/apache//jmeter/binaries/apache-jmeter-2.7.tgz
$md5sum ~/Downloads/apache-jmeter-2.7.tgz
73435baa6ed99c528dacfa36c7e1f119

Configure
$tar -zxvf ~/Downloads/apache-jmeter-2.7.tgz
$sudo mkdir -p /usr/lib/jmeter/2.7
$sudo mv apache-jmeter-2.7/* /usr/lib/jmeter/2.7/
$rm -r apache-jmeter-2.7
$/usr/lib/jmeter/2.7/bin/jmeter.sh

Install git on ubuntu

This is a brief step by step guide to install git and hook up to github using ssh.

Get git
$sudo apt-get install git-core

Check the install
$which git
$git --version

Configure git
$git config --global user.name "Firstname Lastname"
$git config --global user.email you@domain.com

Set up your ssh key
$ssh-keygen -t rsa -C "you@domain.com"
[hit enter]
set your key password
Your identification has been saved in /home/username/.ssh/id_rsa.
Your public key has been saved in /home/username/.ssh/id_rsa.pub.
The key fingerprint is:
fingerprint you@domain.com

Add the key to github
$less ~/.ssh/id_rsa.pub
Copy the key to your clipboard

Go to your github Account Settings

Click "SSH Keys" in the left sidebar
Click "Add SSH key"
Paste your key into the "Key" field
Click "Add key"
Confirm the action by entering your GitHub password

Verify the key
$ssh -T git@github.com
Y
enter the key password

You should get a success, "but GitHub does not provide shell access".

Set up a test repo
$mkdir repo-name
$cd repo-name
$git init
$touch README
$git add README
$git commit -m 'first commit'
$git remote add origin https://github.com/gitusername/repo-name.git
$git push -u origin master

you'll get a 403 error

Getting past the 403 error
$vi .git/config
change https for ssh
$git push -u origin master

you'll get a Permission denied (publickey) error

hmmm

$ssh-add -l shows the same fingerprint as the account settings on github
$ssh -vT git@github.com comes back saying I have successfully authenticated...

Getting past the Permission denied error
$cd ~/.ssh
$vi config

Host github.com
User git
Port 22
Hostname github.com
IdentityFile ~/.ssh/id_rsa
TCPKeepAlive yes
IdentitiesOnly yes

$git push -u origin master

Happy days! You should now see the README file on github

A life ban from Stackoverflow

So, today I decided I'll spend an hour looking at StackOverflow and helping out where I could. It has come to the rescue on a few occasions in the past so I thought I'd return the favour.

I signed up and started looking at the > 3 million questions. Bound to be something in there I can help with. I looked at the newest questions and clicked through. Many of the questions are poorly phrased, some aren't even asking a question and others just want the code; they don't want to do any thinking, they don't want to write the code, they just want the code sent to them.

So I tackled a few questions, some on mongodb, some on jquery, some of java and some javascript.

Some question gets answered so quickly, so you have to be quick. The reason is your post can be deleted if it is deemed a duplicate answer and deletion count against you. There were a couple of occasions where there was less than a minute between my answer and a similar answer were posted and one of them being deleted as a duplicate. 

As a new user I cannot post comments. I didn't think anything of it at the time, so I put my requests for further information in the answer boxes. I had a few responses and started to earn a few points. Then I started to get lambasted for not answering the question and just asking more questions. I've always thought it is difficult to answer a question if it is unclear what is being asked. Eventually, whilst posting an answer to a mongodb question and the 16M BSON limit I received the message stating my account was now banned from posting and would never be reinstated; a life ban.

So basically, I had to earn 50 points before I can comment. But I can't earn any points until I understand the question or there is at least enough information in the question for it to be answerable, or at least until it is actually a question.

Apparently this has already been discussed here

I guess that is the good deed of the day shot down in flames. I'll think twice before I attempt to help again.

So, should we boycott Stack Overflow?

Hell no! It has saved my life on a couple of occasions. They could/should however make easier for people (esp. those new and/or under 50 points) to ask for clarity in poorly phrased questions, or in some instances ask the asker what it is they are actually asking.