[2019 oracle code one] JPMS Modules with Gradle

Building JPMS Modules with Gradle

Speakers: Paul Bakker @pbakker

For more blog posts, see The Oracle Code One table of contents


General

  • Nobody in room yet using module system
  • Often takes several years after a feature release to be adopted.
  • Took 2-3 years after lambda use was common
  • Most open source provides a real module or at least a module name in the meta-inf

Plugin

  • Maven supported module system early on and still does.
  • Gradle did not so developed a plugin
  • https://github.com/java9-modularity/gradle-modules-plugin

IntelliJ

  • Shows modules as folders

Gradle

  • Each module has own build.gradle
  • Root build.gradle file is where the modular plugin is applied
  • external plugin so set classpath and apply to all java projects
  • Dependencies are defined in both build.gradle and module-info.java. Have to do this in Maven too.
  • Plugin adds gradle configuration when compile/test.

Flags

  • Showed exports – plugin has addExports option when compile
  • Showed opens – plugin has addOpens option when compile
  • Also supports AddModules and addReads like the Java options

Application plugin

  • Creates tar/zip with lib and bin folder
  • Supports module path

Packages

  • Split packages
  • “The classpath always works – as long as you don’t have expectations”
  • Always a problem; now in your face
  • patch-module as workaround. Puts library in patchlibs instead of being on module path

Multiple start scripts

  • Out of box, can create one zip file/one starting point
  • Module plugin allows creating custom startup scripts

Testing

  • For whitebox testing, ok to break encapsulation.
    • Classpath is ok
    • Plugin provides moduleOptions on test to choose
  • Black box testing – want module path to test how module works from outside module boundaries.
    • Use module path
    • Need additional module for this approach for tests

Java versions

  • plugin has modularity.mixedJavaReleases
  • compiler runs in two steps
  • compiles everything module-info.java using recent java version (presumably Java 9) and pulls out module-info.java
  • compiles other sources using java 8
  • the module-info.java is ignored by Java 8

My take

I found the gradle files hard/impossible to read (red/purple/green) on dark background. That made it hard for me to follow the parts I didn’t already know. I did learn some though so it was worth going. And it’s great that gradle has a JPMS plugin! I like how he mapped the command line to Gradle code.

jenkins convert to pipeline plugin

I needed to convert a freestyle job to a pipeline job for “Automating your Ci/CD Stack with Java and Groovy” Hands On Lab at Oracle Code One. It’s a really simple project, but it is still a pain to convert by hand.

I had read about the Convert to Pipeline plugin so decide to give it a shot. It’s easy enough to install a plugin.

Generated Pipeline

Below is what got generated. While I’d like to assume that credentials binding isn’t in wide use, that one didn’t surprise me. In hindsight, it makes sense because the credentials binding would need to wrap the Gradle block. I was surprised that Gradle wasn’t supported.

I am glad that the tool leaves comments for what you need to do by hand.

// Powered by Infostretch 

timestamps {

node () {

	stage ('osprey - Checkout') {
 	 checkout([$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: '', url: 'https://github.com/boyarsky/OracleCodeOne2018-HOL-Automating-Stack-Groovy.git']]]) 
	}
	stage ('osprey - Build') {
 	
// Unable to convert a build step referring to "org.jenkinsci.plugins.credentialsbinding.impl.SecretBuildWrapper". Please verify and convert manually if required.
// Unable to convert a build step referring to "hudson.plugins.gradle.Gradle". Please verify and convert manually if required.
		// JUnit Results
		junit 'osprey-project/build/test-results/test/*.xml' 
	}
}
}

Using the pipeline generator

On the pipeline job, you can click the “pipeline syntax” link. It opens a new browser tab with a pull down for common pipeline operations.

Credentials

I choose “withCredentials”, entered the variable names and selected the credential from the pull down. Then I clicked “Generate Pipeline Script” and got:


withCredentials([usernamePassword(credentialsId: 'nexus', passwordVariable: 'nexusPassword', usernameVariable: 'nexusUserName')]) {
// some block
}


Gradle

That brings us to Gradle. As far as I can tell, there are only objects for Gradle if you are using Artifactory. I am using the gradle wrapper though, so I followed the advice of this Stack Overflow post and just called gradlew.


withCredentials([usernamePassword(credentialsId: 'nexus', passwordVariable: 'nexusPassword', usernameVariable: 'nexusUserName')]) {
  sh "./osprey-project/gradlew clean build -b osprey-project/build.gradle -PnexusBaseUrl=http://nexus:8081 -PnexusUserName=${nexusUserName} -PnexusPassword=${nexusPassword}"
}

I decided to separate the publish step into a different stage and rename the stages as well.  Resulting in:


// Powered by Infostretch 

timestamps {

node () {

	stage ('Checkout') {
 	 checkout([$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: '', url: 'https://github.com/boyarsky/OracleCodeOne2018-HOL-Automating-Stack-Groovy.git']]]) 
	}
	stage ('Build') {
	    
	   withCredentials([usernamePassword(credentialsId: 'nexus', passwordVariable: 'nexusPassword', usernameVariable: 'nexusUserName')]) {
          sh "./osprey-project/gradlew clean build -b osprey-project/build.gradle -PnexusBaseUrl=http://nexus:8081 -PnexusUserName=${nexusUserName} -PnexusPassword=${nexusPassword}"
       }

		// JUnit Results
		junit 'osprey-project/build/test-results/test/*.xml' 
	}
	
	stage ('Publish') {
	    
	   withCredentials([usernamePassword(credentialsId: 'nexus', passwordVariable: 'nexusPassword', usernameVariable: 'nexusUserName')]) {
          sh "./osprey-project/gradlew publish -b osprey-project/build.gradle -PnexusBaseUrl=http://nexus:8081 -PnexusUserName=${nexusUserName} -PnexusPassword=${nexusPassword}"
       }
	}
}
}

Note: My actual build file was bigger. The parts outside the pipeline (ex: project security settings) got preserved.

building motion profile generator with gradle and/or eclipse

Last year, I blogged about how to build Smart Dashboard with Gradle for FRC. Today, a student asked me how to build Motion Profile Generator using gradle. It’s similar, but seemed like a good time to update the post for this specific project.

You can use pure command line when reading/following this. Alternatively, you can use Eclipse as little or as much as you like. I include the Eclipse “gotchas.” (Spoiler: It’s faster to just use the command line)

Pulling from git

You can pull the code from git in any way you’d like. Common ways are:

  • From the command line (or git bash): git clone https://github.com/vannaka/Motion_Profile_Generator.git
  • From Eclipse, go to the git perspective, and paste in the URL.
  • Download the project from github as a zip file. (Please don’t choose this option. It is better to use git properly. The only reason I’ve ever done the zip option is if I need to look at a code on a computer where I can’t install anything.)

Running Gradle at the command line

The discussion was centered around building Motion Profile Generator. If you aren’t involved with a FIRST Robotics Competition team, this is used to help with autonomous programming. As with many Gradle projects, the project comes with Gradle right in the project. So all you have to is:

  1. cd to directory you cloned Motion Profile Generator in
  2. chmod 700 gradlew
  3. ./gradlew build

Make sure you have internet before running this the first time. If you want to build directly from Eclipse, see my Eclipse Gradle post and skip to the BuildShip section.

Where is the output when building SmartDashboard?

Running gradle creates a build directory. You can look at it on the file system. Or if you are using Eclipse,

  1. Configure the filters so show the Gradle build folder
  2. Add/remove the JRE library as described here to get rid of the Eclipse errors about access restrictions.
  3. Hit refresh before looking for it.

I don’t care how gradle works, I just want to run the program!

No  worries. Just:

  1. cd build/libs
  2. java -jar Motion_Profile_Generator-v3.0.0.jar

Ok now I’m curious, what gets generated?

This build directory  contains a number of things:

  • distributions – a zipped up and tarred up version of Motion Profile Generator which includes the jar file, all need dependencies and the script to kick it off.
  • classes – the compiled .class files
  • libs – a jar file with the compiled code in this project
  • resources – the JavaFX file (user interface files)
  • scripts – to launch Motion Profile Generator
  • tmp – would you really expect to find something important in a directory named tmp?

What does the Gradle build file do?
Gradle is a build tool. The build.gradle file in Motion Profile Generator contains Groovy code that declares what the build should do. Both Gradle and Maven follow “convention over configuration” which means that a lot is implied! I went over most (but not all) of this out loud in the classroom.

A plugin is like a helper tool that Gradle is going to use in order to build. For example, ‘java’ is going to compile among other things.  Most of these plugins are in common use. Gradle needs to download files in order to run. This includes some default plugins. These files are hosted in a repository for binary files. A really common one is Maven Central. This is used by both Maven and Gradle builds to obtain binaries. This build file is really simple. It says that Gradle needs to know how to build a standalone java application.

plugins {
    id 'application'
    id 'java'
}

This tells Java to put some information in the manifest. In particular, which class should run automatically when you try to run the jar. It also includes a version number for reference.

mainClassName = "com.mammen.main.Main"
version = "v3.0.0"

Remember the convention over configuration thing I mentioned earlier? Convention says the source code is in the nested directories src/main/java. This project decided to put the source code directly in the src directory. Therefore the project needs to inform Gradle of this non-standard location.

sourceSets {
    main.java {
        srcDirs = ['src']; include '**/*.java'
    }

    main.resources {
        srcDirs = ['src']; exclude '**/*.java'
    }
}

Normally, Gradle projects include references to dependencies on the internet. This project decided to supply them directly in a lib directory. Again, a non-standard thing to inform Gradle about. (I think this decision is because Pathfinder has extra files)

dependencies {
    compile fileTree(dir: 'lib', include: '*.jar')
}

Next the script specifies that we should create a jar file and an uber jar file including the special files..

jar {
    from configurations.compile.collect { it.isDirectory() ? it : zipTree(it) }
    // Set up manifest to point the main class to the right class.
    manifest {
        attributes 'Main-Class': mainClassName
        attributes 'Version': version
    }
	from ("lib/native_libs") {
		into ("")
	}
}

The end! The very last thing is to declare which version of Gradle this entire script should be run with.

task wrapper(type: Wrapper) {
    gradleVersion = '3.3'
}

Output from building in gradle

I’m including the output from a “good” run in case you have problems and want to compare. This is for the first time you run it. After that the download steps won’t be in the output and it will run much faster.

JeanneBrskysMBP:Motion_Profile_Generator nyjeanne$ ./gradlew build
Downloading https://services.gradle.org/distributions/gradle-4.7-bin.zip
.......................................................................

Welcome to Gradle 4.7!

Here are the highlights of this release:
 - Incremental annotation processing
 - JDK 10 support
 - Grouped non-interactive console logs
 - Failed tests are re-run first for quicker feedback

For more details see https://docs.gradle.org/4.7/release-notes.html

Starting a Gradle Daemon (subsequent builds will be faster)

> Task :compileJava
/Users/nyjeanne/git/Motion_Profile_Generator/src/com/mammen/main/ProfileGenerator.java:4: warning: OutputFormat is internal proprietary API and may be removed in a future release
import com.sun.org.apache.xml.internal.serialize.OutputFormat;
                                                ^
/Users/nyjeanne/git/Motion_Profile_Generator/src/com/mammen/main/ProfileGenerator.java:5: warning: XMLSerializer is internal proprietary API and may be removed in a future release
import com.sun.org.apache.xml.internal.serialize.XMLSerializer;
                                                ^
/Users/nyjeanne/git/Motion_Profile_Generator/src/com/mammen/main/ProfileGenerator.java:435: warning: OutputFormat is internal proprietary API and may be removed in a future release
            OutputFormat format = new OutputFormat(dom);
            ^
/Users/nyjeanne/git/Motion_Profile_Generator/src/com/mammen/main/ProfileGenerator.java:435: warning: OutputFormat is internal proprietary API and may be removed in a future release
            OutputFormat format = new OutputFormat(dom);
                                      ^
/Users/nyjeanne/git/Motion_Profile_Generator/src/com/mammen/main/ProfileGenerator.java:439: warning: XMLSerializer is internal proprietary API and may be removed in a future release
            XMLSerializer xmlSerializer = new XMLSerializer(
            ^
/Users/nyjeanne/git/Motion_Profile_Generator/src/com/mammen/main/ProfileGenerator.java:439: warning: XMLSerializer is internal proprietary API and may be removed in a future release
            XMLSerializer xmlSerializer = new XMLSerializer(
                                              ^
Note: /Users/nyjeanne/git/Motion_Profile_Generator/src/com/mammen/ui/javafx/MainUIController.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
6 warnings

BUILD SUCCESSFUL in 24s
6 actionable tasks: 6 executed