3-2: Variant & Artifact API v1

You might be scratching your head, "Since API v1 is outdated, why not dive straight into API v2?" Well, the reason is that most of the existing posts and queries are tied to version v1. So, I've decided to begin with the API version that's more familiar to most, contrasting v1 and v2 in various aspects, highlighting v2's enhancements and strengths. This step-by-step approach will aid in comprehending the scripts and Plugins of your current project, laying a robust groundwork for transitioning to v2. Bear in mind, the terms "API v1/v2" are coined just for this book to make things easier for you; the official Android terminology is "New Variant/Artifact API" for the v2 version.

Below is a common AGP extension configuration snippet, relevant only to the Android Application module, our following code will be based on it.

// The basic configuration
defaultConfig {
  applicationId = "me.xx2bab.extendagp.variantapi"
  minSdk = 28
  targetSdk = 31
  versionCode = 1
  versionName = "1.0"

  testInstrumentationRunner = "androidx.test.runner.AndroidJUnitRunner"
}

// Configure two default build types
buildTypes {
  getByName("debug") {
    isMinifyEnabled = false
    applicationIdSuffix = ".debug"
  }
  getByName("release") {
    isMinifyEnabled = true
    proguardFiles(
      getDefaultProguardFile("proguard-android.txt"),
      "proguard-rules.pro"
    )
  }
}

// Add two new flavors under "server" dimensions
flavorDimensions += "server"
productFlavors {
  create("staging") {
    dimension = "server"
    applicationIdSuffix = ".staging"
    versionNameSuffix = "-staging"
  }
  create("production") {
    dimension = "server"
    applicationIdSuffix = ".production"
    versionNameSuffix = "-production"
    versionCode = 2
  }
}

You can find the source code mentioned in this section in the variant-api project.

3-2-1: Variant - Fetching Configured Data

We've learned buildType and productFlavor fuse to form the essence of the Variant. At this juncture, Variant content falls into two classifications: resolved and merged configuration data and Variant-Aware TaskProvider.

val android = project.extensions.getByType(AppExtension::class.java) [I]
android.applicationVariants.configureEach { [II]
  val variant: ApplicationVariant = this
  val variantCapitalizedName = variant.name.capitalize()
  
  // Configurations (Mirror the DSL models)
  logger.lifecycle("variant name: ${variant.name}")
  logger.lifecycle("variant.applicationId:" 
    + "${variant.applicationId}")
  logger.lifecycle("variant.versionCode: "
    + "${variant.versionCode}")
  logger.lifecycle("variant.mergedFlavor.minSdkVersion:"
    + variant.mergedFlavor.minSdkVersion)
  logger.lifecycle("variant.mergedFlavor.manifestPlaceholders:"
    + "${variant.mergedFlavor.manifestPlaceholders}")
  (variant.mergedFlavor as MergedFlavor).applicationId = 
    variant.applicationId + ".custom"

  // Task Providers
  val beforeAssemble = project.tasks.register(
    "before${variantCapitalizedName}Assemble"
  ) {
    doFirst { 
      logger.lifecycle("${this.name} is running...") 
    }
  }
  variant.assembleProvider.configure {
    dependsOn(beforeAssemble)
  }  …
}

At point [I], the AppExtension is the android {...} extension configuration you see in scripts, and configureEach at [II] stems from the DomainObjectCollection interface function, iterating each element according to the container's constituents. In the core logic:

• You can tap into the merged applicationId and other fundamental data from ApplicationVariant, more advanced configurations, like manifestPlaceholder, via variant.mergedFlavor.
• When crafting custom Tasks, exported Task Providers such as variant.assembleProvider that can be utilized for dependency orchestration to insert our Tasks into existing build process. As illustrated, we injected a tailored beforeAssemble Task preceding the Variant-Aware assemble Task.

3-2-2: Variant - The Variant Aware Configuration

Previously in Section 3-1, we touched on the cascading sequence of productFlavor -> buildType -> defaultConfig. This mechanism is designed to curtail redundant code but can't cater to all configuration needs. Suppose both staging and release have signingConfig(...) set. In that case, the staging configuration prevails, but if we desire a specific test signature instead of staging, we must employ the following advanced configuration method:

android.applicationVariants.configureEach {
  …
  if (variant.name.contains("release", true)
   && variant.name.contains("staging", true)
) {
  (variant.mergedFlavor as MergedFlavor).setSigningConfig(...)
}

The variant.mergedFlavor returns a ProductFlavor interface type, lacking some useful APIs, and certain properties inside are immutable, such as val applicationId: String?, so we usually cast it to MergedFlavor before proceeding. The advanced configuration is versatile with the Variant API, it enables more specific configuration, acting on the final Variant objects instead of buildType{...} or productFlavor{...} as referred in static configuration.

3-2-3: Variant - Disabling Specific Combinations

In Android build systems, Variants are formed as the Cartesian product of buildTypes and productFlavors. However, sometimes certain Variant combinations might be generated that are unnecessary for your project. To handle these, you can filter them out using the variantFilter(...) method.

android.variantFilter {
  if (name == "stagingRelease") {
    ignore = true
  }
}

By setting the ignore property to true for the Variant named stagingRelease, you disable that particular combination. It will no longer be included in the applicationVariants collection, so subsequent configurations and Tasks for that combination won't be generated. This can speed up the Configuration phase significantly when a bunch of unused combinations get ignored.

3-2-4: Artifact - Acquisition

Next, we'll dig into the Artifact aspect. What can we expect from its related APIs?

android.applicationVariants.configureEach {
  …
  variant.outputs.forEach { output ->
    val file = output.outputFile
    if (file.extension == "apk") {
      …
    }
  }
}

Within Variant API v1, the Artifact API isn't clearly detailed in the documentation. Yet, you can see that the "outputs property" serves to retrieve the final product, grouped as variant.outputs. For an Android Application module, you can fetch the APK (but not the AAB) file, and for a Library module, you can obtain the AAR. Remember, at the configuration stage, these are non-existent File objects (they are not readable), often passed into the Task configuration.

Let's take a look at a simple Task: "Renaming an APK". First, create the RenameApkFile Task class to define the basic input and output.

abstract class RenameApkFile : DefaultTask() {
  @get:InputFile
  lateinit var inputApk: File

  @get:OutputFile
  lateinit var outputApk: File

  @TaskAction
  fun taskAction() {
    inputApk.copyTo(outputApk)
  }
}

Next, create the Task based on the outputFile of the APK. Pass in the source file, the target file, and add a dependency to the packageApplicationProvider. There's no need to rely on the assembleProvider only to get the APK, as Assemble Tasks are mainly lifecycle hooks devoid of logic (Task Action).

variant.outputs.forEach { output ->
  val file = output.outputFile
  if (file.extension == "apk") {
    val out = File(file.parentFile, "custom-${variant.versionName}")
    val renameApkTask = project.tasks.register<RenameApkFile>(
        "rename${variantCapitalizedName}Apk") {
      inputApk = file
      outputApk = out
      dependsOn(variant.packageApplicationProvider)
    }
    …
  }
}

Here, the method to rename APKs, output.outputFileName, has been replaced with a custom Task for copying and renaming, as it was removed after AGP 7.0.

3-2-5: Key Points of Crafting AGP Synergy Plugins

In the realm of Gradle Plugins, Tasks that depend on the AGP API are referred to AGP Synergy Plugins within this book, with two essential elements: input artifacts and Task dependencies.

• Input artifacts: Extending AGP functionality requires raw artifacts from AGP, such as APK, AndroidManifest, resource files, etc.
• Task dependencies: The timing of obtaining and processing these artifacts is crucial. Managing these dependencies ensures minimal impact on the original process.

The above Task of consuming the APK is a special case in the API v1 scenario, as we can easily acquire the artifacts. However, determining which Task to rely on can be difficult. One approach is to understand the Android build process, then experiment with CLI modes and use commands like --dry-run and --scan to analyze Task execution and dependencies.

:app:clean SKIPPED
:app:beforeProductionDebugAssemble SKIPPED
:app:preBuild SKIPPED
:app:preProductionDebugBuild SKIPPED
...
:app:packageStagingDebug SKIPPED
:app:renameStagingDebugApk SKIPPED
:app:apkSizeObtainStagingDebug SKIPPED
:app:notifyStagingDebugBuild SKIPPED
:app:createStagingDebugApkListingFileRedirect SKIPPED
:app:assembleStagingDebug SKIPPED
:app:assembleDebug SKIPPED

A more typical scenario might be "accessing the merged AndroidManifest.xml for custom manipulation", such as altering the allowBackup attribute. If you were to abstract this logic as an independent Task, the code would look something like this:

abstract class ManifestAfterMergeTask : DefaultTask() {

  @get:InputFile
  var mergedManifest: File  // [I]
  
//  @get:OutputFile
//  var modifiedManifest: File  // [II]

  @TaskAction
  fun afterMerge() {
    val modifiedManifest = mergedManifest.readText()
      .replace("allowBackup=\"true\"", "allowBackup=\"false\"")
    // Overwrite the merged manifest
    mergedManifest.writeText(modifiedManifest) 
  }

}

In this process, the main consideration is how to fetch the mergedManifest file. To get to this point, you'll have to sift through the Task list to find those related to the "Manifest". Below is a list obtained with --dry-run:

:app:createStagingDebugCompatibleScreen**Manifest**s SKIPPED
:app:extractDeepLinksStagingDebug SKIPPED
:app:processStagingDebugMain**Manifest** SKIPPED
:app:processStagingDebug**Manifest** SKIPPED
:app:processStagingDebug**Manifest**ForPackage SKIPPED

A careful review of various Task implementations led us to discover that the ProcessApplicationManifest class is the one beyond the processStagingDebugMainManifest Task name. This class contains the mergedManifest, marked with @get:OutputFile — the exact merged AndroidManifest.xml that we want.

Next, you might wonder how to insert custom logic into this process. There are three common approaches.

Firstly, the Task approach with finalizedBy(...): The finalizedBy(...) API in Gradle is an option because it permits a subsequent Task to run immediately after the intended one. Warning however! Similar to Java's try...finally... construct, this one wasn't built for Task chaining in mind (you will see more in Section 4-4). This is a possible misuse, but it works:

val processManifestTask = project.tasks
  .withType(ProcessApplicationManifest::class.java)
  .first { it.name.contains(variant.name, true) }
val postUpdateManifestTask = project.tasks
  .register("postUpdate${variantCapitalizedName}Manifest",
    ManifestAfterMergeTask::class.java) {
    mergedManifest = processManifestTask.mergedManifest
      .get()
      .asFile
  }
// Misuse of finalizedBy()
processManifestTask.finalizedBy(postUpdateManifestTask)

Secondly, the Task approach with insertion strategy.: Task approach again, but this time put it between two consecutive AGP Tasks, A and B. It's done through postUpdateManifestTask.dependsOn(taskA) and taskB.dependsOn(postUpdateManifestTask). You can verify these Tasks through visualization using the --scan command.

val processManifestTask = project.tasks
  .withType(ProcessApplicationManifest::class.java)
  .first { it.name.contains(variant.name, true) }
val postUpdateManifestTask = project.tasks
  .register("postUpdate${variantCapitalizedName}Manifest",
    ManifestAfterMergeTask::class.java) {
    mergedManifest = processManifestTask.mergedManifest
      .get()
      .asFile
    // Prerequisite Task  
    dependsOn(processManifestTask)
  }
project.tasks
  .withType(ProcessMultiApkApplicationManifest::class.java)
  .first { it.name.contains(variant.name, true) }
  .dependsOn(postUpdateManifestTask) // Post-task

Since our goal is to modify the merged Manifest file—a fixed position in the Android App build process, arbitrary modification might lead subsequent Manifest-related Tasks astray. Thus, we don't define the output file in the ManifestAfterMergeTask task but have to overwrite the input file. However, be warned: this will cause the caching mechanism of the task to fail.

... is not up-to-date because: \
  Task has not declared any outputs despite executing actions.

The mutual modifications between the production Task (ProcessApplicationManifest) and the in-place modification Task (ManifestAfterMergeTask) will invalidate the cache, as illustrated in the flowchart (Figure 3.2.1).

None of the alternatives presented in the second solution can meet the current scenario's requirements (in-place modification), as they all interrupt the Gradle caching mechanism. If a new Task doesn't alter an Android Gradle Plugin (AGP) intermediate product in-place but rather creates a different product file, the strategy remains applicable.

The third and most recommended option is to use doLast(...) to append a post Task Action to the Task. This method is pretty standard, yet it still has the downside of being closely tied to the original Task.

If you're dealing with additional input or output parameters or files, the Gradle Task Runtime API must be employed to dynamically add them. For instance:

// Analogous to @get:Input, invalidating the cache if the property changes
processManifestTask.inputs
  .property("replace_value", "allowBackup=\"false\"")

// Other commonly used APIs:
// inputs.file(...) / files(...)
// outputs.file(...) / files(...)

doFirst(...) or doLast(...)make it easy to insert our custom logic when compared to the first and second methods.

// Alteration point
processManifestTask.doLast("postUpdateAction") { 
  val task = this as ProcessApplicationManifest
  val targetValue = task.inputs.properties["replace_value"]
    .toString()
  // Acquire the input artifact and change it
  val modifiedManifest = task.mergedManifest.get()
    .asFile
    .readText()
    .replace("allowBackup=\"true\"", targetValue)
  task.mergedManifest.get().asFile.writeText(modifiedManifest)
}

To summarize the methods of injecting custom logic into the AGP Synergy Plugin, we typically rely on the three above-mentioned techniques when executing commands like assembleXxx or bundleXxx to trigger the automated workflow. For a more in-depth conversation about task orchestration, don't miss Section 4-4.

3-2-6: Facing the Challenges of API v1

With API v1, the Artifact aspect only uncovers the final product of AGP (found in ${appOrLibModule}/build/outputs), preventing us from accessing any intermediate products. To introduce further customizations, developers need to master the skill of authoring Tasks in AGP Synergy Plugins and grasp the common practices of collaborating Plugins with AGP, such as retrieving intermediate products via Task inputs and outputs.

However, this is just the tip of the iceberg. The real complexity often lies in uncovering the input artifacts, which might be nestled within private methods or other private AGP components. Creating a simple Task extension becomes a quest, where the effort spent on inspecting the source code often exceeds the actual designing and coding, leading to risky techniques like reflection and forced type conversion. The peril here is that a minor AGP upgrade could break your hook.

The root of this complexity is that Gradle Tasks aren't suited as public APIs, therefore make it difficult for Synergy Plugin development. The decoupling of external interfaces from internal implementations becomes hard, and AGP's continual evolution inevitably alters its internal Tasks, creating hurdles for third-party collaboration and extension. This challenge was a significant motivator for the advancement to API v2.

Moreover, the early Gradle API showed limitations, offering less methods of data transmission between Tasks. Since AGP v0.1 came out in 2013 with Android Studio and Gradle was at version 1.x at the time, they were limited by what the platform could do. A classic example: if I need an Int or String type of input, calculated at the execution phase and not passed into our custom Task in the configuration phase, what's the best strategy? Early Gradle API solutions were generally file-container-based, involving file IO consumption and a complicated intermediate data process, sometimes even necessitating non-maintainable Task hooks.

In Section 4-2, we'll explore the improvement brought by the Provider<T> API to tackle this issue, and the ensuing API v2, built upon the Provider<T> API's concept of Lazy Configuration, promises a more user-friendly experience.

3-2-7: In Summary

As the Variant and Artifact API v1 approaches obsolescence after years of service, we reflect on the challenges and complexities it brought to third-party AGP Synergy Plugins and their interactions. But we also acknowledge how it paved the way for mastering API v2. It's a lesson in evolution, one that emphasizes the need for continuous adaptation and improvement.