using spring RestTemplate and 2 factor authenication to add issues to github

Last month, I figured out how to use Spring’s RestTemplate to add issues to github programmatically figuring it would save me time in the future.  Well, the future is here.  I needed to add 16 issues (one per milestone.)  I run my program and get a 401.  In particular, I got

Apr 20, 2014 6:42:18 PM org.springframework.web.client.RestTemplate handleResponseError

WARNING: GET request for "https://api.github.com/repos/boyarsky/repoName/issues" resulted in 401 (Unauthorized); invoking error handler

Exception in thread "main" org.springframework.web.client.HttpClientErrorException: 401 Unauthorized

at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:91)

at org.springframework.web.client.RestTemplate.handleResponseError(RestTemplate.java:588)

at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:546)

at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:502)

at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:444)

at jb.Trial2.main(Trial2.java:29)

Right.  I’ve changed my password AND enabled two factor authentication on github in the past month.  Luckily, it was easy to switch my program to two factor.  Here’s what I did.

Create personal use token

I created a second personal use token just like I did for for my git commmand line use by going to the applications page and clicking “generate new token”.  I chose to create a separate token so I could revoke access as soon as I’m done running the web service.

Testing the token

Per the getting started with oauth doc, I ran a command line to test the token:

curl -i -H 'Authorization: token notMy40CharToken' \

https://api.github.com/user

 Wrote another trial program

I adapted my second trial program from the initial iterations to test out two factor.

package jb;

import org.springframework.http.*;
import org.springframework.web.client.*;

/**
 * Call a web service that uses authenticated user to test passing credentials
 *
 * @author jeanne
 *
 */
public class Trial3 {

	public static void main(String[] args) {
		String url = "https://api.github.com/repos/boyarsky/oca-ocp-book/issues";

		HttpHeaders headers = new HttpHeaders();

		String personalToken = "notMy40CharToken";
		headers.add("Authorization", "token " + personalToken);

		// ignore result because just testing connectivity
		HttpEntity<String> request = new HttpEntity<String>(headers);
		RestTemplate template = new RestTemplate();
		template.exchange(url, HttpMethod.GET, request, Object[].class);
		System.out.println("success");
	}

}

I then replaced the authentication part of my real program and it worked like a charm.  Even with the changes to the program, it was faster than creating 16 issues by hand with the proper text/assignee/milestone.

Delete the personal token

I deleted the token to ensure it never gets used again.  I don’t want to run the program by accident with my credentials.  Or accidentally post the token here.

using spring RestTemplate to add issues to github

I’m using github as a project management tool for a (non-work) project.  That’s been working well for being able to track things and have it close to the project artifacts.  Milestones have dates (to remember what was committed to) and issues correspond to tasks or follow up items within the milestone.  I reached a point where I needed to add the same three tasks to over 15 milestones.  (And possibly more in the future.)

It probably would have taken an hour to do this manually.  It’s a boring task so I imagine I’d have gotten distracted.  The boredom of even contemplating the task plus the possibility that I might have to do again make me realize I should spend a little longer and write code.  Which meant the prep time was a little over an hour but the time to run the program was less than a minute.

I worked in iterations because I wanted to learn early on if this was feasible (or a time sink.)

Iteration 1 – setup a project and call a github web service

I used Maven because I knew I’d need a couple libraries and dependencies.  In Eclipse, I created a new Maven project with

  • group id = jb
  • artifact = github-issue-creator

I then added one dependency.  I knew I’d need Spring Web because I wanted to use RestTemplate.  At this point, my pom.xml looks like

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>jb</groupId>
  <artifactId>github-issue-creator</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <dependencies>
    <dependency>
      <groupId>org.springframework</groupId>
      <artifactId>spring-web</artifactId>
      <version>4.0.2.RELEASE</version>
    </dependency>
  </dependencies>
</project>

It was then time to actually call RestTemplate.  I quickly reviewed the docs to confirm my memory of the flow.  Then I wrote the simplest program I could that would call a Github web service.  I had learned from the GitHub API docs of the Octocat Hello World test repository.   I confirmed the issues query web service command in the GitHub docs and went to https://api.github.com/repos/octocat/Hello-World/issues in a browser.  Perfect – I get JSON back.

I then wrote the RestTemplate code using this URL.

package jb;

import org.springframework.web.client.*;

/**
 * View a public repo's issue via a web service on github to make sure can
 * connect. This particular repo is referenced int he documentation.
 *
 * @author jeanne
 *
 */
public class Trial1 {

	public static void main(String[] args) {
		String url = "https://api.github.com/repos/octocat/Hello-World/issues";

		RestTemplate template = new RestTemplate();

		// ignore result because just testing connectivity
		template.getForObject(url, Object[].class);
		System.out.println("success");
	}

}

You’ll notice I’m telling Spring to return an array of Objects.  I don’t need a more specific class since I’m ignoring the result.  It’s just a proof of concept – my real code will do a POST and own’t care about getting the data back.  I knew an array was needed because the JSON in the browser returned a list.  If you do need to deal with the returned values, look at this Jackson example with Spring and RestTemplate.

Iteration 2 – use my credentials to access a github web service

So far so good.  I have two more steps left to accomplish my goal.  I can either test passing credentials or try a POST.  I decided to test passing credentials first so I would be doing harmless GET operations during the testing.

I’ve only used basic authentication with RestTemplate once so I had to look up what to do.  This blog post was helpful showing how to use the Apache Commons Codec library to deal with credentials.

First, I had to add this dependency to my POM.

<dependency>
  <groupId>commons-codec</groupId>
  <artifactId>commons-codec</artifactId>
  <version>1.9</version>
</dependency>

This code obtains all the issues for myRepoName.  And no, that isn’t my GitHub password.  It isn’t my repository name for that matter.

package jb;

import org.apache.commons.codec.binary.*;
import org.springframework.http.*;
import org.springframework.web.client.*;

/**
 * Call a web service that uses authenticated user to test passing credentials
 *
 * @author jeanne
 *
 */
public class Trial2 {

	public static void main(String[] args) {
		String url = "https://api.github.com/repos/boyarsky/myRepoName/issues";

		String plainCreds = "boyarsky:notMyGitHubPassword";
		byte[] plainCredsBytes = plainCreds.getBytes();
		byte[] base64CredsBytes = Base64.encodeBase64(plainCredsBytes);
		String base64Creds = new String(base64CredsBytes);

		HttpHeaders headers = new HttpHeaders();
		headers.add("Authorization", "Basic " + base64Creds);

		// ignore result because just testing connectivity
		HttpEntity<String> request = new HttpEntity<String>(headers);
		RestTemplate template = new RestTemplate();
		template.exchange(url, HttpMethod.GET, request, Object[].class);
		System.out.println("success");
	}

}

As you can see, I encode the bytes of my username/password combination. I then set that as the basic authentication header. I had to switch to “exchange” instead of “getForObject” as noted in that blog post for this to work.

Iteration 3 – actually creating an issue

There are only four differences between this and iteration 2 from a web services perspective.

  1. From the github create issue docs, I learned the POST URL is/repos/:owner/:repo/issues.
    private static final String URL = “https://api.github.com/repos/boyarsky/myRepoName/issues”;
  2. Use HttpMethod.POST instead of HttpMethod.GET.  I actually wasted some time with this step.  I forgot to change it so the program was running without error and returning  issues rather than setting them.
  3. Pass in the JSON for the values you want to set in the issue.
  4. Now I tell Spring to excpect an Object instead of an array because the JSON returned is no longer an array.
String json = "{\"title\": \"" + title + "\", \"assignee\": \""
				+ assignee + "\", \"milestone\": " + milestone + "}";

		// ignore result because just testing connectivity
		HttpEntity<String> request = new HttpEntity<String>(json, headers);
		RestTemplate template = new RestTemplate();
		template.exchange(URL, HttpMethod.POST, request, Object.class);

I tested this once and it worked. I then added the logic to get the proper values (it is formulaic) and let it fly.

Getting the milestone id was easy – it is in the URL: https://github.com/boyarsky/myRepoName/issues?milestone=4&state=open

Conclusion
While it would have been faster to add all the issues by hand, this was certainly more interesting and more fun. And the next time I need to add an issue to each milestone, it will be trivial. Thanks github for the wonderful documentation!

 

edit: if you need to do this for two factor authentication, see my updated post

Rod Johnson in the cloud – the server side java symposium

Rod Johnson led off with a Dilbert cartoon about the cloud/platform – the pointy haired boss says “it’s like you are a technologist and philosopher all in one” in response to “blah blah blah cloud”.

Benefits of cloud

  • No reason to operate own data center because does not scale. Also noted the environmental impacts due to overprovisioning on small/medium data centers. Large companies will adopt onsite/internal clouds.
  • Compared computing to a utility and noted factories used to operate their own electricity. Really good analogy!

Perceived weakness for Java in the cloud

  • Perceived to be slow and startups look to Ruby and the like. This is a problem because these are the companies that are tomorrow’s leaders
  • Universal hosting not available like PHP

Strengths of Java in the cloud

  • Write once run anywhere
  • Java already in the enterprise
  • In actuality scales better than Ruby

Other Points

  • We think if the internet as free when we use it. However, there is a cost and the energy use is very high.
  • Rod Johnson considers flexibility to be a more important driver than cost. That way pay when need and not guessing up front if a startup will succeed. Compared to the Zipcar model. It costs more per mile but cheaper if don’t drive a lot. Another great analogy.
  • Simplicity, because large organizations have a lot of bureaucracy and provides a way to subvert that bureaucracy [some of that bureaucracy has a reason, accounting and security teams are not going to be happy with subverting things. However, an internal cloud provides benefits without subverting.]
  • Need architectural trade offs for economies of scale
  • If cloud is about business agility, need to be able apps fast which is why Rails shines. Notes the importance of being perceived as being able to start fast. [sounds like this is more valuable than maintenance]
  • It is more important to have one good option than a lot of medium options. Thinks Java community needs a cultural change to pick a stack. Showed how having two IDES of eclipse and intelij means good products and not a lot of poor choices. Interesting NetBeans was left off the list. rails helps because gives you less choices and saves time by encouraging consistency
  • The line between development and operations is blurring. No common tool chain organizational chaos. More self service in cloud because developers deploy

Spring’s thoughts

  • IaaS (infrastructure as a service) – vmware vcloud
  • PaaS (software as a service) – must abstract away OS, provide self service and be a productive programming model
  • Frameworks are key to portability – gave example that Spring isolates you from the app server [in other words if you deploy your container in your application, you don’t have to worry about the external container as much]. I liked the Ruby example better because Rails is. Framework that doesn’t replace the container.
  • Challenges: a lot more and complex data, adapt to RDBMS + NOSQL. Recognize what datastores need to be transactional. plugged Spring data project. Similarly for asynchronous data patterns
  • Visual builders are usually horrible because build uneditable code. Wavemaker is different because Spring owns it. I think there as another reason he tried to state but i didn’t catch it. Maybe it builds on roo?

Java’s phases

  • -2005 : make java work
  • 2005-2010: RIA, REST, higher productivity
  • 2010-?: beyond RDBMS, mobile/tablet, cloud, even higher productivity required

the end was all blatant advertising of spring’s tools.