using spring RestTemplate and 2 factor authenication to add issues to github

Last month, I figured out how to use Spring’s RestTemplate to add issues to github programmatically figuring it would save me time in the future.  Well, the future is here.  I needed to add 16 issues (one per milestone.)  I run my program and get a 401.  In particular, I got

Apr 20, 2014 6:42:18 PM org.springframework.web.client.RestTemplate handleResponseError

WARNING: GET request for "https://api.github.com/repos/boyarsky/repoName/issues" resulted in 401 (Unauthorized); invoking error handler

Exception in thread "main" org.springframework.web.client.HttpClientErrorException: 401 Unauthorized

at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:91)

at org.springframework.web.client.RestTemplate.handleResponseError(RestTemplate.java:588)

at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:546)

at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:502)

at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:444)

at jb.Trial2.main(Trial2.java:29)

Right.  I’ve changed my password AND enabled two factor authentication on github in the past month.  Luckily, it was easy to switch my program to two factor.  Here’s what I did.

Create personal use token

I created a second personal use token just like I did for for my git commmand line use by going to the applications page and clicking “generate new token”.  I chose to create a separate token so I could revoke access as soon as I’m done running the web service.

Testing the token

Per the getting started with oauth doc, I ran a command line to test the token:

curl -i -H 'Authorization: token notMy40CharToken' \

https://api.github.com/user

 Wrote another trial program

I adapted my second trial program from the initial iterations to test out two factor.

package jb;

import org.springframework.http.*;
import org.springframework.web.client.*;

/**
 * Call a web service that uses authenticated user to test passing credentials
 *
 * @author jeanne
 *
 */
public class Trial3 {

	public static void main(String[] args) {
		String url = "https://api.github.com/repos/boyarsky/oca-ocp-book/issues";

		HttpHeaders headers = new HttpHeaders();

		String personalToken = "notMy40CharToken";
		headers.add("Authorization", "token " + personalToken);

		// ignore result because just testing connectivity
		HttpEntity<String> request = new HttpEntity<String>(headers);
		RestTemplate template = new RestTemplate();
		template.exchange(url, HttpMethod.GET, request, Object[].class);
		System.out.println("success");
	}

}

I then replaced the authentication part of my real program and it worked like a charm.  Even with the changes to the program, it was faster than creating 16 issues by hand with the proper text/assignee/milestone.

Delete the personal token

I deleted the token to ensure it never gets used again.  I don’t want to run the program by accident with my credentials.  Or accidentally post the token here.

using spring RestTemplate to add issues to github

I’m using github as a project management tool for a (non-work) project.  That’s been working well for being able to track things and have it close to the project artifacts.  Milestones have dates (to remember what was committed to) and issues correspond to tasks or follow up items within the milestone.  I reached a point where I needed to add the same three tasks to over 15 milestones.  (And possibly more in the future.)

It probably would have taken an hour to do this manually.  It’s a boring task so I imagine I’d have gotten distracted.  The boredom of even contemplating the task plus the possibility that I might have to do again make me realize I should spend a little longer and write code.  Which meant the prep time was a little over an hour but the time to run the program was less than a minute.

I worked in iterations because I wanted to learn early on if this was feasible (or a time sink.)

Iteration 1 – setup a project and call a github web service

I used Maven because I knew I’d need a couple libraries and dependencies.  In Eclipse, I created a new Maven project with

  • group id = jb
  • artifact = github-issue-creator

I then added one dependency.  I knew I’d need Spring Web because I wanted to use RestTemplate.  At this point, my pom.xml looks like

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>jb</groupId>
  <artifactId>github-issue-creator</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <dependencies>
    <dependency>
      <groupId>org.springframework</groupId>
      <artifactId>spring-web</artifactId>
      <version>4.0.2.RELEASE</version>
    </dependency>
  </dependencies>
</project>

It was then time to actually call RestTemplate.  I quickly reviewed the docs to confirm my memory of the flow.  Then I wrote the simplest program I could that would call a Github web service.  I had learned from the GitHub API docs of the Octocat Hello World test repository.   I confirmed the issues query web service command in the GitHub docs and went to https://api.github.com/repos/octocat/Hello-World/issues in a browser.  Perfect – I get JSON back.

I then wrote the RestTemplate code using this URL.

package jb;

import org.springframework.web.client.*;

/**
 * View a public repo's issue via a web service on github to make sure can
 * connect. This particular repo is referenced int he documentation.
 *
 * @author jeanne
 *
 */
public class Trial1 {

	public static void main(String[] args) {
		String url = "https://api.github.com/repos/octocat/Hello-World/issues";

		RestTemplate template = new RestTemplate();

		// ignore result because just testing connectivity
		template.getForObject(url, Object[].class);
		System.out.println("success");
	}

}

You’ll notice I’m telling Spring to return an array of Objects.  I don’t need a more specific class since I’m ignoring the result.  It’s just a proof of concept – my real code will do a POST and own’t care about getting the data back.  I knew an array was needed because the JSON in the browser returned a list.  If you do need to deal with the returned values, look at this Jackson example with Spring and RestTemplate.

Iteration 2 – use my credentials to access a github web service

So far so good.  I have two more steps left to accomplish my goal.  I can either test passing credentials or try a POST.  I decided to test passing credentials first so I would be doing harmless GET operations during the testing.

I’ve only used basic authentication with RestTemplate once so I had to look up what to do.  This blog post was helpful showing how to use the Apache Commons Codec library to deal with credentials.

First, I had to add this dependency to my POM.

<dependency>
  <groupId>commons-codec</groupId>
  <artifactId>commons-codec</artifactId>
  <version>1.9</version>
</dependency>

This code obtains all the issues for myRepoName.  And no, that isn’t my GitHub password.  It isn’t my repository name for that matter.

package jb;

import org.apache.commons.codec.binary.*;
import org.springframework.http.*;
import org.springframework.web.client.*;

/**
 * Call a web service that uses authenticated user to test passing credentials
 *
 * @author jeanne
 *
 */
public class Trial2 {

	public static void main(String[] args) {
		String url = "https://api.github.com/repos/boyarsky/myRepoName/issues";

		String plainCreds = "boyarsky:notMyGitHubPassword";
		byte[] plainCredsBytes = plainCreds.getBytes();
		byte[] base64CredsBytes = Base64.encodeBase64(plainCredsBytes);
		String base64Creds = new String(base64CredsBytes);

		HttpHeaders headers = new HttpHeaders();
		headers.add("Authorization", "Basic " + base64Creds);

		// ignore result because just testing connectivity
		HttpEntity<String> request = new HttpEntity<String>(headers);
		RestTemplate template = new RestTemplate();
		template.exchange(url, HttpMethod.GET, request, Object[].class);
		System.out.println("success");
	}

}

As you can see, I encode the bytes of my username/password combination. I then set that as the basic authentication header. I had to switch to “exchange” instead of “getForObject” as noted in that blog post for this to work.

Iteration 3 – actually creating an issue

There are only four differences between this and iteration 2 from a web services perspective.

  1. From the github create issue docs, I learned the POST URL is/repos/:owner/:repo/issues.
    private static final String URL = “https://api.github.com/repos/boyarsky/myRepoName/issues”;
  2. Use HttpMethod.POST instead of HttpMethod.GET.  I actually wasted some time with this step.  I forgot to change it so the program was running without error and returning  issues rather than setting them.
  3. Pass in the JSON for the values you want to set in the issue.
  4. Now I tell Spring to excpect an Object instead of an array because the JSON returned is no longer an array.
String json = "{\"title\": \"" + title + "\", \"assignee\": \""
				+ assignee + "\", \"milestone\": " + milestone + "}";

		// ignore result because just testing connectivity
		HttpEntity<String> request = new HttpEntity<String>(json, headers);
		RestTemplate template = new RestTemplate();
		template.exchange(URL, HttpMethod.POST, request, Object.class);

I tested this once and it worked. I then added the logic to get the proper values (it is formulaic) and let it fly.

Getting the milestone id was easy – it is in the URL: https://github.com/boyarsky/myRepoName/issues?milestone=4&state=open

Conclusion
While it would have been faster to add all the issues by hand, this was certainly more interesting and more fun. And the next time I need to add an issue to each milestone, it will be trivial. Thanks github for the wonderful documentation!

 

edit: if you need to do this for two factor authentication, see my updated post