Best Practices for Reproducible Research

Session 8

Ingmar Steiner

2017-06-28

Gradle Plugins

Plugin recap

Inline class

We can write a custom plugin inline in the build script

build.gradle

class CountdownPlugin implements Plugin<Project> {
  void apply(Project project) {
    project.task("countdown") {
      doLast {
        println "Executing $name"
      }
    }
  }
}

apply plugin: CountdownPlugin

buildSrc subproject

Plugin code can also be provided in the buildSrc

buildSrc/src/main/groovy/CountdownPlugin.groovy

import org.gradle.api.*

class CountdownPlugin implements Plugin<Project> {
  void apply(Project project) {
    project.task("countdown") {
      doLast {
        println "Executing $name"
      }
    }
  }
}

build.gradle

apply plugin: CountdownPlugin

Standalone “object plugin”

countdown-plugin/settings.gradle

rootProject.name = 'countdown-plugin'

countdown-plugin/build.gradle

plugins {
  id 'java-gradle-plugin'
  id 'groovy'
}

gradlePlugin {
  plugins {
    countdownPlugin {
      id = 'countdown'
      implementationClass = 'CountdownPlugin'
    }
  }
}

countdown-plugin/src/main/groovy/CountdownPlugin.groovy (see above)

Object plugin consumer project

countdown-project/build.gradle

buildscript {
  dependencies {
    classpath name: 'countdown-plugin'
  }
}

apply plugin: 'countdown'

Composite build:

Object plugin publish/resolve

Less agile, but easier to use:
  1. Apply Maven publishing to plugin
  2. Resolve plugin from repository as buildscript classpath dependency
Even less agile, but easiest to use:
  1. Publish object plugin to Gradle Plugins Portal (or elsewhere)
  2. Use plugins DSL in consumer project
  3. (If resolved from elsewhere, use pluginRepositories block in settings.gradle)

Testing

Testing frameworks

Automatic software testing frameworks exist for

Java
Python

etc.

TestNG unit test example

src/main/java/Foo.java

public class Foo {
  public static String foo() {
    return "foo";
  }
}

src/test/java/FooTest.java

import org.testng.*;
import org.testng.annotations.*;
public class FooTest {
  @Test
  public void testFoo() {
    String expected = "foo";
    String actual = Foo.foo();
    Assert.assertEquals(expected, actual);
  }
  @Test
  public void testBar() {
    Assert.assertTrue(false);
  }
}

Running tests with Gradle

build.gradle

apply plugin: 'java'
repositories.jcenter()
dependencies.testCompile 'org.testng:testng:6.9.6'
test.useTestNG()

Testing Plugins

Gradle TestKit

Plugin code

src/main/groovy/CountdownPlugin.groovy

import org.gradle.api.*
class CountdownPlugin implements Plugin<Project> {
  void apply(Project project) {
    project.task("countdown", type: CountdownTask)
  }
}

src/main/groovy/CountdownTask.groovy

import org.gradle.api.DefaultTask
import org.gradle.api.tasks.*

class CountdownTask extends DefaultTask {
  @Input
  int from = 3

  @TaskAction
  void countdown() {
    (from..1).each {
      print "$it... "
    }
    println "BLAST-OFF!!!"
  }
}

Plugin functional test example

src/test/groovy/CountdownPluginTest.groovy

import org.gradle.testkit.runner.GradleRunner
import org.testng.annotations.*

class CountdownPluginTest {
  @Test
  void testCountdown() {
    def tmpDir = File.createTempDir()
    new File("$tmpDir/build.gradle").withWriter {
      it.println 'plugins {'
      it.println "  id 'countdown'"
      it.println '}'
    }
    def gradle = GradleRunner.create().withProjectDir(tmpDir).withPluginClasspath()
    def result = gradle.withArguments('countdown').build()
    assert result.output.contains('3... 2... 1... BLAST-OFF!!!')
  }

  @Test
  void testCountdownWithCustomFrom() {
    def tmpDir = File.createTempDir()
    new File("$tmpDir/build.gradle").withWriter {
      it.println 'plugins {'
      it.println "  id 'countdown'"
      it.println '}'
      it.println 'countdown {'
      it.println '  from = 5'
      it.println '}'
    }
    def gradle = GradleRunner.create().withProjectDir(tmpDir).withPluginClasspath()
    def result = gradle.withArguments('countdown').build()
    assert result.output.contains('5... 4... 3... 2... 1... BLAST-OFF!!!')
  }
}

End-to-end Example

Overview

  1. Download speech synthesis dataset (from festvox)
  2. Unpack audio files from **/*.wav
  3. Unpack label files from **/*.lab
  4. Unpack and convert text prompts from txt.done.data
  5. Concatenate audio files to a single FLAC file (use Praat, SoX, or FFmpeg)
  6. Store prompts, utterance times, segment durations in a single YAML file
  7. Publish FLAC and YAML (“FLAML”)
  8. Refactor this as a plugin
  9. Test plugin functionality

Demo time!

https://bitbucket.org/psibre/bestpract-flaml

Assignment

Reverse, FLAML “extraction” plugin

  1. Develop plugin which adds build logic to any project
  2. Resolve specified FLAC+YAML file pair as data dependencies
  3. Extract utterances from YAML to text and label files
  4. Extract utterance audio from FLAC to wav files
Due date
2017-07-12 (two weeks from today)

Questions?