Over time, the system topology, or landscape, here has grown and grown… we now have a lot of systems (and components within them) and it’s not always clear what depends on what (which is a very important part of our deployment planning process). Dependencies can be divided into two distinct categories: build dependencies and runtime dependencies. This blog will focus on build dependencies.
Gradle is our build tool of choice while JFrog’s Artifactory is our artifact repo. Here I’ll present a custom gradle task that produces a build receipt (which is, in turn, published to Artifactory providing us with a record of the build (along with the jars, wars, and other artifacts associated with the component being built). We ultimately import each and every build receipt into a PostgreSQL database which, together with a browser-based UI, provides us with a very clear picture of what depends on what (and a whole lot more). As such, our deployment planning is a much simpler process.
Cutting to the chase, the build receipt is a json representation of a component’s build dependencies for each gradle configuration (e.g. compile, runtime, etc.). Here’s a snippet which shows that the component in question depends upon spring-web, httpclient, and jaxb-core, et al, for the compile configuration. Furthermore, the snippet shows that, because of transitive dependencies, two distinct versions of spring-web were requested while one version was selected (by gradle).
{
"dependencies": [
{
"configuration": "compile",
"group": "org.springframework",
"name": "spring-web",
"requestedVersions": [
"4.1.9.RELEASE",
"3.2.4.RELEASE"
],
"version": "4.1.9.RELEASE"
},
{
"configuration": "compile",
"group": "org.apache.httpcomponents",
"name": "httpclient",
"requestedVersions": [
"4.4.1"
],
"version": "4.4.1"
},
{
"configuration": "compile",
"group": "com.sun.xml.bind",
"name": "jaxb-core",
"requestedVersions": [
"2.2.11"
],
"version": "2.2.11"
},
.
.
.
]
}
Herein lies the BuildReceiptTask (which, as you will see, leverages the gradle ResolutionResult)…
package com.example.scm.gradle.plugins.task
import java.util.List;
import java.util.Map;
import java.util.Set;
import groovy.json.JsonBuilder
import groovy.json.JsonOutput
import org.gradle.api.DefaultTask
import org.gradle.api.Project
import org.gradle.api.artifacts.component.ComponentIdentifier
import org.gradle.api.artifacts.component.ComponentSelector
import org.gradle.api.artifacts.component.ModuleComponentIdentifier
import org.gradle.api.artifacts.component.ModuleComponentSelector
import org.gradle.api.artifacts.result.DependencyResult
import org.gradle.api.artifacts.result.ResolutionResult
import org.gradle.api.artifacts.result.ResolvedDependencyResult
import org.gradle.api.logging.Logger
import org.gradle.api.logging.Logging
import org.gradle.api.tasks.OutputFile
import org.gradle.api.tasks.TaskAction
/**
* Captures dependencies for the project being built.
*/
class BuildReceiptTask extends DefaultTask {
/**
* Adds the buildReceipt task.
*
* @param project the gradle project.
*/
public static void addBuildReceiptTask(Project project) {
project.task('buildReceipt', type: BuildReceiptTask) {
group 'Build'
description 'Captures dependencies'
}
project.artifacts {
archives project.tasks.buildReceipt.outputFile
}
project.assemble.dependsOn project.buildReceipt
project.uploadArchives.dependsOn project.buildReceipt
}
/**
* The output file.
*/
@OutputFile
def File outputFile
/**
* Default constructor.
*/
public BuildReceiptTask() {
filename = "${project.buildDir}/build-receipt/build-receipt.json"
outputFile = new File(filename)
}
/**
* The gradle task action (invoked when the buildReceipt task is
* executed).
*/
@TaskAction
void performTask() {
msg = "[BuildReceiptTask] Create a build receipt including dependencies"
project.logger.info msg
List<Dependency> dependencies = getDependencies(project)
BuildReceipt buildReceipt = new BuildReceipt(dependencies)
writeBuildReceipt(buildReceipt)
}
/**
* Captures dependencies (for each and every configuration).
*
* @param project gradle project
*/
List<Dependency> getDependencies(Project project) {
List<Dependency> dependencies = new ArrayList<Dependency>();
project.configurations.each { conf ->
ResolutionResult rr = conf.incoming.resolutionResult
Set<DependencyResult> dependencyResults =
rr.getAllDependencies().findAll()
if (dependencyResults.size() > 0) {
Map<ModuleComponentIdentifier, Set<ModuleComponentSelector>> dependencyMap =
getDependencyMap(dependencyResults)
dependencyMap.each{ componentIdentifier, componentSelectorSet ->
String group = componentIdentifier.getGroup()
String module = componentIdentifier.getModule()
String version = componentIdentifier.getVersion()
List<String> requestedVersions = new ArrayList<String>()
componentSelectorSet.each{ componentSelector ->
requestedVersions.add(componentSelector.getVersion())
}
Dependency dependency = new Dependency(requestedVersions,
conf.name, group, module, version)
dependencies.add(dependency)
}
}
}
return dependencies
}
/**
* Returns a map containing each and every "selected" dependency and their
* associated "requested" components.
*
* @param dependencyResults the dependency results (per gradle)
* @return dependency map
*/
Map<ModuleComponentIdentifier, Set<ModuleComponentSelector>> getDependencyMap(Set<DependencyResult> dependencyResults) {
Map<ModuleComponentIdentifier, Set<ModuleComponentSelector>> dependencyMap =
new HashMap<ModuleComponentIdentifier, Set<ModuleComponentSelector>>();
dependencyResults.each { dr ->
ResolvedDependencyResult rdr = (ResolvedDependencyResult) dr
ComponentIdentifier componentIdentifier = rdr.selected.getId()
project.logger.debug("[BuildReceiptTask] Selected dependency... " +
componentIdentifier.toString());
if (componentIdentifier instanceof ModuleComponentIdentifier) {
ComponentSelector componentSelector = rdr.requested
// Note, the component selector set holds the "requested"
// components.
Set<ComponentSelector> componentSelectorSet =
dependencyMap.get(componentIdentifier);
if (componentSelectorSet == null) {
componentSelectorSet = new HashSet<ComponentSelector>();
}
componentSelectorSet.add(componentSelector)
dependencyMap.put(componentIdentifier, componentSelectorSet)
}
}
return dependencyMap;
}
/**
* Writes the build receipt to the task's output file.
*/
void writeBuildReceipt(BuildReceipt buildReceipt) {
String jsonOutput = JsonOutput.toJson(buildReceipt)
String formattedJsonOutput = JsonOutput.prettyPrint(jsonOutput)
outputFile.withWriter { out ->
out.write(formattedJsonOutput)
}
}
class BuildReceipt {
List<Dependency> dependencies
BuildReceipt(List<Dependency> dependencies) {
this.dependencies = dependencies
}
}
class Dependency {
List<String> requestedVersions
String configuration
String group
String name
String version
Dependency(List<String> requestedVersions, String configuration,
String group, String name, String version) {
this.requestedVersions = requestedVersions
this.configuration = configuration
this.group = group
this.name = name
this.version = version
}
}
}
Tom Muldoon
Software Architect