I had a brief chat earlier today about your standard CTF experience at xyzSecCon. It's always been a bit biased towards breaking-in rather than defending, and biased towards networks/services/apps/etc rather than on just the app layer.
Some initial thoughts for a potentially interesting and new(er) format are thus:
There are teams of attackers
There are teams of defenders
Each defending team is given the same application to defend
* The application is riddled with issues, you name it, it's there
Each defending team has the same codebase with which to start from
Each defending team is given a branch within the SCM repo
* the build pipeline is in place
* dependencies, project files and imports to different IDEs (eclipse, intellij) are all there
* VMs are auto provisioned and releases are deployed to those VMs
* essentially all the defenders have to think about is the code, everything else is in place
Each branch and build track has the same tool set,
* functional and non-functional unit tests including some security centric unit tests (xUnit, etc),
* functional and non-functional user tests including some security centric user tests (Selenium or casper js, etc),
* static code analysis (Sonar, Findbugs, Coverity, Fortify, etc),
* automated web scanning (acunetix, burp, skipfish, etc)
Each attacking team is given prior knowledge of all the flaws within said application
* well in advance if required/to make it more fun
Each attacking team can use any tool they see fit - this is about attack the app, not the network/service/OS
* In all cases the attacking teams can add their own tools to the tool set
Each defending team must deploy at least once an hour
Each deployment goes to a new environment for that team - so if the CTF/DTF lasts over 2 (10 hour) days, then each team needs 20 different virtual envs.
* This is so that any long running attacks have time to complete against any given 'release'
The application contains your standard web functionality
* Anonymous/Authenticated browsing
* Registration
* Login/Logout
* Forgotten password
* Common issues such as Captcha/Username already registered/etc
* Register CC
* Calls out to payment providers/etc
* Shopping cart
* Checkout
* RSS feeds
* Social network integration
* etc etc, this is just a quick list
One (of many) responses to consider
* If there is a leakage of passwords - for instance - the defending team need to think about how they flick a switch so that all new logins are forced to change a password
Obviously the attacking teams will find most vulns early on, so the idea is that the defending teams reduce the number of exploitable vulns over time. Attacking teams can attack all the deployments for all the defending teams, so the scoring is recorded in a matrix
There can even be a set of dumb users; they can be socially engineered on an automagic basis - ie links in emails coming into email inboxes are automagically clicked
Anyway, I've thought about this for a total of an hour or so, but I think there is something different in this, something different to your normal run-of-the-mill CTF
It's usually easier to break stuff than it is to fix it. The initial challenge is in fixing rather than breaking.
Once the defending teams have gone through several release iterations, it will be harder to break stuff than it is to fix it. The longer challenge is in breaking rather than fixing.
Points will be awarded for the number of vulnerabilities detected; there will be a sliding scale for the level to which the vulnerability is exploited. For instance, displaying alert(1) on an attackers browser is 5 points, but harvesting the application user base session ids would be worth 50 points.
Different attacking teams will adopt different strategies. Do they go after the low hanging fruit that the defending teams will likely implement defences for in the first few releases; thus picking up some early points to add to their total but potentially wasting time developing exploits for the trickier issues that the defending team will not implement fixes for until release 10?
