csrf – extending the owasp solution and “interesting” IE javascript bugs (part 2)

While implementing CSRF for JForum, I needed to extend the OWASP solution.  Let me tell you, they don’t make it easy to extend.  Lots of final.  Here’s what I did – linked to code on github.

To read about the original problem or why I choose the OWASP filter, see part 1.

Extending the OWASP solution

  1. CsrfFilter.java – The OWASP filter is final so had to copy the logic in my own.  I added logic to get the action method name/
  2. CsrfHttpServletRequestWrapper.java – Since I’m using actions instead of URLs, I need to make the request look as if it the actions are URLs.  A short simple class.
  3. CsrfListener.java – OWASP assumes you are using URLs and you can enumerate them in the property file.  I have action names and there a lot of pages to allow as unprotected – without wildcard patterns to help, this is unwieldy.  The OWASP listener isn’t final but the logic I needed to extend was in the middle of a method.  So I copied their listener adding a method that reads the csrf.properties and creates “org.owasp.csrfguard.unprotected.” properties for all lines that weren’t set to “AddToken.”
  4. Redirect.java – the OWASP Redirect class works just fine if you always have the same context root.  We have a different one when testing locally vs on the server.  And since the path is in a property file, it isn’t easy to change.  Of course the OWASP class was final so I had to copy logic rather than extending it.  My Redirect class gets the server and context root from the request and adds it to the relative path in the OWAP property file.
  5. AddJavaScriptServletFilter.java – Adds the OWASP JavaScript to each page after </head> unless it is a download or a user isn’t logged in.  Since logged in users can’t do anything CSRF exploitable, they don’t need the token.  This also makes Google happier because googlebot doesn’t see the extra parameter confusing the URLs.
  6. AjaxHelper.java – The OWASP filter adds “OWASP CSRFGuard Project” to the X-Requested-With header for AJAX calls.  You can customize it to not include that, but I wasn’t sure what would happen if I left it blank.  It was easy enough to change JForum to use startsWith instead of equals() to make it more flexible.
  7. Owasp.CsrfGuard.js (attempt 1)
    1.  The JavaScript provided by OWASP has a bug.  It goes through the properties in document.all from top to bottom.  As it finds forms, it adds hidden input fields.  So far so good.  The problem is that when you have a lot of forums, some of them get bumped past the original end of the document.  So the loop never sees them and the CSRF token is never added.  The solution is to go through the document from bottom to top.  I’ve submitted a fix to OWASP for this.  What did the trick for understanding the problem was adding console.log(“length: ” +  document.all.length);  at the beginning and end.  It grows by roughly 100.   The bug fix was changing in injectTokens()
      for(var i=len -1; i>=0; i--) {

      to

      // BUG FIX: if go through in ascending order, the indexes change as you add hidden elements
      
      // when have 100 forms, this moves some of the forms past the original document.all size
      
      for(var i=len -1; i>=0; i--) {
    2. IE doesn’t like
      var action = form.getAttribute("action");

      when you have a form field named action.  (If at all possible, DON’T DO THAT!!!).  Since JForum has 100’s of such reference, changing it at this point isn’t an option.  IE helpfully returns some object.  It’s not an array/map or list of any kind that I can tell.  After getting hopelessly stuck on this, I asked Bear Bibeault for help.  He noted the answer was in “Secrets of the JavaScript Ninja” a book he wrote that I had recently read.  And it was.  Solution

      // hack to test if action is a string since IE returns [object] when action in form and as hidden field
      
      // if not a string, assume it is our action and add token for now
      
      var action = form.getAttributeNode("action").nodeValue;
    3. The OWASP code contains this innocent looking code.
      var hidden = document.createElement("input");
      
      hidden.setAttribute("type", "hidden");
      
      hidden.setAttribute("name", tokenName);
      
      hidden.setAttribute("value", (pageTokens[uri] != null ? pageTokens[uri] : tokenValue));

      . It works fine in all browsers except IE. Want to know what IE does? That’s right.  It generates a hidden field with the correct token value and *no* token name.  I found this problem solution online and changed the code to

      
      var hidden;
      
      try {
      
      hidden = document.createElement('<input type="hidden" name="' + tokenName + '" />');
      
      } catch(e) {
      
      hidden = document.createElement("input");
      
      hidden.type = "hidden";
      
      hidden.name = tokenName;
      
      }
      
      hidden.value = (pageTokens[uri] != null ? pageTokens[uri] : tokenValue);
      
      
    4. My last IE change was an easy one.  Easy in that it was a problem I had seen before.
      var location = element.getAttribute(attr);

      changed into

      //var location = element.getAttribute(attr);
      
      // hack - getting same error as on action - don't know why but hack to move forward
      
      var attr = element.getAttributeNode(attr);
      
      var location = null;
      
      if ( attr != null) {
      
      location = attr.nodeValue;
      
      }
      
    5. Then I encountered an issue with the event handling framework in IE>  I didn’t even look at this.  All IE debugging was done between 9pm and midnight January 28th-30th after we all thought CSRF was fine from being in production the previous Sunday.  The only computer I had access to that ran IE is from 2002 and barely runs.  I was tired of IE and didn’t want to throw any more time down that rabbit hole.
  8. Owasp.CsrfGuard.js (attempt 2) – After encountering numerous IE problems, I would up merging the code from OWASP CSRF Guard version 2 and 3 to make it less dependent on the browser.

Finally, we get to look at the JForum specific parts in part 3.

production problems across time zones

A couple days ago, I blogged about the technical details of a production problem (not caused by me) at coderanch.  Now that the problem is resolved, is an interesting time to reflect on how time zones helped us.

Peak volume at the ranch

While we have users from 219 countries, roughly half our volume is from the US and India combined.  (source google analytics)  I also learned that our “peak time” is midnight to 6am Mountain Standard Time followed by 6am to 3pm.  This would be business hours in Asia and Europe followed by Europe and North America.  Peak time is misleading because bots count as users for hits.

As an added bonus, peak time for search engines/bots is 5am to 7am Mountain Standard Time.  Yes, these overlap.

When the problem occurred

Lucky for us, we have a moderator in India (Jaikiran Pai) who was able to investigate the problem real time.  Which mean those of us in the United States woke up to an almost daily email saying that site went down and an attempted fix.

Fixes for other problems

It turned out there were a couple resource leaks in the code that Jaikiran found/fixed.  One had been in the code for over a year.  One was new (due to an API being converted to JPA and the caller not adapting the open session filter.)  One was a less than desirable transaction setting.  All of these manifested because of the new, bigger problem – but were not the cause.  This is a common problem in software – finding the RIGHT problem.

Converging on the right problem

Another advantage of having someone who could look at the problem real time was that he was able to capture the database logs real time.  Right before going to sleep, Jaikiran found two queries taking a long time to run.  And by a long time, I mean one was taking OVER A MINUTE under load.  Which he found by running:

select current_query,now() - pg_stat_activity.query_start as duration from pg_stat_activity order by duration desc

He posted the two queries.  One took 200K explain plan units.  At this point, we had something that could be fixed without witnessing the problem firsthand and sql tuning work moved back to the United States. One thing the *right* solution had that the others didn’t was that it explained everything.  All the other fixes made sense, but relied on a “magic” step to get from the problem to the solution.

Tuning the hack

I created a hack that would limit the # threads shown in a forum to get us through another day or two until the weekend.  It required tuning during the production problem time.  Back to India.

Conclusion

Communication across time zones only worked because of email.  (Normally, we’d have used the forums.  But the forums weren’t a very reliable place given that the problem was the forums going down.)  I’ve never been on a team at work more than 3 time zones away.  It was a great experience working with a strong developer half the world away.  And while we’ve been developing features together, it is what you do in times of difficulty that shows your process.  It was wonderful to see ours working.

And finally: GREAT JOB JAIKIRAN!!!

trying architexa – an eclipse diagramming plugin

Architexa recently announced free licensing for individuals or teams of up to three.  I figured I’d run it against CodeRanch JForum to see what happens.

Requirements/Install

The software is an Eclipse plugin.  You get the link after registering.  A different update site is provided for Eclipse 3.X vs Juno (4.2).  I know some CodeRanch JForum developers use IntelliJ IDEA.  I don’t, so I can try it out.  The install was smooth.  I did get asked to confirm I trust the certificate within Eclipse.

One minor discrepancy.  The website says the software is free for individuals and teams of up to three developers.  The email confirming your email and welcoming you says the software is free for individuals and teams of up to four developers.  Moot point at the moment since I’m using it as an individual.  But Architexa should sync these up.

The email asks me to validate my email using a localhost link.  Same for editing my profile.  I can’t click on it to validate.  Hm.. I then went to the website and clicked the “forgot password” link.  This got an emailed password which I could use to validate in Eclipse itself.  Again the change password link is a localhost link.  After entering that password, Eclipse said my account was validated.  So while the links are broken, I’m in the tool.

Learning curve

Architexa asks which projects it should index.  I said just JForum.  Architexa provides good Eclipse “cheat sheets” to start out quickly.

Layered Diagram

This is like a dependency graph for packages.  Very nice.  If you mouse over, you see incoming (afferent) and outgoing (efferent) dependencies.  You can also drill down to see lower level packages.

Class Diagram

Right clicking a package opens the option to create a class diagram.  Two classes generated on top of each other, but I can drag them around (or highlight them or call other attention.)  It is easy to view the source code from the class diagram.  I don’t see how to view the method names/fields in the class diagram.  This info is available in the outline view in Eclipse already though so it isn’t critical.

Sequence Diagram

This appears to be an enhanced editor.  You drag classes into it and it shows calls.  This seems like more work to create.  I tried dragging a few items over and “add all” to get the calls.  Unfortunately calls aren’t so much within one or two classes so this didn’t help much.  I didn’t create anything worth taking a screenshot of.

Overall opinion

The package diagram caught my attention the most.  I really like the dependency arrows.  The class diagram provides a nice visualization as well.  The sequence diagram seems like it would be a good documentation aid if one was creating sequence diagrams for the project.  Which I’m not because we inherited the design of the code and I’m already familiar with the flow.  I think more value would come from sharing documents and using the tool as a team.

For a “real” (paid) project, I”m not sure I’d be so thrilled to keep my documentation in a proprietary tool. Even a free one.  But for getting a feel for the software on a product that is “documentation-lite” or “no documentation”, the layered (package) diagrams and class diagrams provide a nice way to jump in.  Assuming you are using Eclipse already of course.