4-2: Leveraging Lazy Properties in Plugin Extensions and Tasks

Recall our foray into the introductory project from Section 2-4. You might remember spotting properties within the plugin's extension and task. But these were straightforward, or more specifically, Direct Properties. Now that the spotlight turns to the Provider and Property Lazy Configuration containers that are widely adopted post Gradle 4.0, properties that are wrapped by the above containers are considerated Lazy Properties. Here is a brief overview of these two types of properties:

Property type	Evaluation	When to use
Direct property	Eager	When the property is first accessed
Lazy property	Lazy	When the property is needed

Through this section, you'll grasp their essence, the weight they carry in the AGP's Variant and Artifact API, and the rationale behind such an implementation.

You'll find the source code nestled in the slack project. Here's a quick breakdown:

• The Plugin project resides at slack/plugins/slack-lazy.
• Tinker with the Plugin configuration at slack/plugin-config/src/main/kotlin/slack-lazy-config.gradle.kts.
• The Android Application test project, a bearer of the Plugin, resides in slack/app. Remember to activate the appropriate Plugin in app.gradle.kts during testing.

4-2-1: Direct Property

Direct Property refers to the original type that a property field intends to express or carry, without the encapsulation of Provider and Property in the Gradle realm. Here's a glimpse:

abstract class SlackNotificationExtension {
  var enabled = true
  var message = ""
  ...
}

abstract class SlackNotificationTask : DefaultTask() {
  
  @get:Input
  var message: String = ""

  @get:InputFiles
  var artifacts: Set<File>? = null

  @get:OutputFile
  var notifyPayloadLog: File? = null
  
  ...
}

Every property here, from enabled to token, or any String, Int, Boolean, Uri, Set<T> type declaration in this context, fits the bill of a Direct Property type. Why should that concern you? Imagine you're a third-party Plugin developer set to craft a tlack Plugin based on the existing slack plugin. You aim to autogenerate the message field in the Extension of the slack, spruced up with the build result's (APK) filename and size. There are two probable solutions:

1. Tweak the message within the Extension: This two-step venture demands pinpointing when message gets its value (options like androidExtension.onVariants(...) or project.afterEvaluate(...) are your allies). Next, ensure that tlack tweaks the message just before task registration. Thank heavens for the delayed configuration API of tasks.register(...), simplifying this!

2. Alter the Task's message: Find that sweet spot after the message assignment but before the task execution (hint: project.afterEvaluate(...)). Next, dive into the TaskProvider via project.tasks.named(...) or project.tasks.withType<T>(...), assigning a fresh message.

Observing the above, we unearth three problems:

1. Absolutely, they could work. But figuring out the best time to make changes? That's a dangerous slope, especially when you have to deal with complicated situations where tasks depend on each other.
2. If the Extension doesn't have the field message, it leads you to the second option. The catch is this: Changing the message (Direct Property type) of a Task won't change its original value. If the original message instance could be used for other Tasks, you might miss some of them and cause things to go in ways you didn't expect.
3. As for message (a String), it's tacitly set during the Configuration phase in our context. However, in real-world scenarios, one might need this value in the Execution phase. Consider a situation like fetching a permission list from a merged AndroidManifest.xml, that's out of reach during the Gradle Configuration.

These issues did not appear overnight. They have persisted for years. In the past, determined developers devised a variety of strategies to address them. Adopting file-centric approaches over Direct Property types and utilizing File, FileCollection handles for parameter transfers (as touched upon in Section 3-2) are cases in point.

4-2-2: Lazy Property

To address the aforementioned issue, one common solution is lazy retrieval, a term that may seem familiar if you've ever used the Dagger2 dependency injection framework on Android. Dagger2 employs Lazy<T> to wrap up dependencies, ensuring an on-demand initialization, thus evading initialization deadlocks.

Similarly, Gradle offers the Provider<T> and Property<T> interfaces, giving a deferred value fetch tied to a specific Task's execution. This delays the calculation of a property’s value, allowing developers to focus on their logic rather than the nuances of when parameter values get initialized. Coupled with map(...) and flatMap(...) operations, they can also broaden the horizons for Task dependency transfers.

A Dive into Provider and Property

Let's dissect the Provider and Property basics.

A Provider signifies a read-only value:

• Access the current value with Provider.get().
• Create it via project.provider(...), or derive it using methods like Provider.map(...) or Provider.flatMap(...).
• Various extensions of Provider exist in Gradle, such as the Task Delayed Configuration API: project.tasks.register(...). The resulting TaskProvider is a descendant of Provider.

On the other hand, Property is a read-write value:

• It extends from Provider.
• Use Property.set(T) to explicitly assign a value to Property<T>, overwriting any prior value.
• There's also Property.set(Provider), allowing you to set a Provider as the value of the Property. This setup lets you connect two Property or Provider instances before the actual value is deduced.
• To create a Property, leverage project.objects.property(Class).

In practical scenarios, Property often represents Extension and Task properties, while parameters like TaskProvider.map(...) or TaskProvider.flatMap(...) output Provider. These lazy property values are live, recalculating their values whenever you invoke Provider.get(). It's a mechanism somewhat analogous to Android Jetpack's LiveData and Transformations.

Let's retrofit the SlackNotificationExtension to employ these lazy properties to tackle challenges such as dynamically generating messages during the Execution phase.

// Refine Extension Property
abstract class SlackNotificationExtension {
  ...
  abstract val message: Property<String>
}

// Inject the Property
abstract class SlackNotificationPlugin : Plugin<Project> {
  ...
  
  val taskProvider = project.tasks.register(
    "assembleAndNotify${this.name.capitalize(Locale.ENGLISH)}",
    SlackNotificationTask::class.java
  ) {
    ...
    message.set(slackExtension.message)
  }
  
  // You can set up the message whenever you want 
  // before the task gets started
  slackExtension.message.set("...")
}

// Retrieve the value in Task Action
abstract class SlackNotificationTask : DefaultTask() {
  ...
  
  @get:Input
  abstract val message: Property<String>
  
  @TaskAction
  fun notifyBuildCompletion() {
    ...
    val customMsg = StringBuilder().append(
      "Message: ${message.get()}\n")
      .append(...)
    artifacts.get().forEach { 
      customMsg.append(it.asFile.absolutePath).append("\n") 
    }
    ...
  }   
}

This updated approach employs the Property to decouple reads and writes. Reading (like in Task Action) doesn't hinge on when or how the message gets initialized or altered. Instead, it focuses on retrieving the latest value at Task Execution phase. Writing is flexible, unbounded by the Task configuration timeline.

Delving into File-related Containers

To bolster build velocities, Gradle has orchestrated a robust build cache (refer to Section 4-5), leaning on serialized file storage. Before our pals Provider and Property arrived on the scene, FileCollection was the go-to file container:

• FileCollection is a versatile file container offering tools for operations like addition, subtraction, search, and filtration.
• It's got a bunch of child classes, including the mutable ConfigurableFileCollection, the hierarchical FileTree, and its mutable variant, ConfigurableFileTree.
• Create a ConfigurableFileCollection with project.files(...) or project.objects.fileCollection(...).
• A neat trick of FileCollection is its ability to implicitly bring Task dependency through the chaining of input and output.

While the above sounds promising, real-world implementation does pose some hitches:

• Ambiguous content: With a FileCollection that could be a single file, a directory, or both, you often need to ascertain its type before use. Methods like getSingleFile() can return a File object when it's a single file, but if it's not, it throws an IllegalStateException.
• Rigid Transfer Mode: Suppose you are going to combine two third-party tasks, the task A's Input relies on task B's Output, and task B spits out a FileCollection while A expects a String. It becomes complicated because you merely need content from B's b.json. Bridging this requires an additional Task for conversion.

Given these challenges, Gradle introduced the FileSystemLocation interface in version 4.x, with its notable sub-interfaces, RegularFile and Directory. FileSystemLocation furnishes a singular method, getAsFile(), which returns a JDK's File object. The nuances of creating these instances are further detailed under their respective sub-interfaces.

Directory designates a folder category, with DirectoryProperty acting as its mutable Property form. To generate a new DirectoryProperty instance, you can use project.objects.directoryProperty(). Notably, project.objects yields an ObjectFactory object—a utility overseen and auto-injected by Gradle. In many scenarios, we prefer to work with project.layout (returning ProjectLayout) to fetch prevalent Directory and DirectoryProperty instances:

public interface ProjectLayout {

  // The project's foundational directory
  Directory getProjectDirectory();
  DirectoryProperty getBuildDirectory();
  
  // Further methods that convert `java.io.File` 
  // into Gradle-specific file containers
  Provider<Directory> dir(Provider<File> file);
  Provider<RegularFile> file(Provider<File> file);
  FileCollection files(Object... paths);
  
}

Subsequently, for other relative paths from the above Directory instances (obtained from ProjectLayout), one can use Directory#dir(String). For securing Provider<Directory>, there's Directory#dir(Provider<? extends CharSequence>). In the same vein, DirectoryProperty offers similar methods to obtain either Provider<Directory> or DirectoryProperty.

As for RegularFile, it characterizes conventional file types, encompassing text, images, or executable files. Distinctively, it's neither a folder nor a Unix device file (node) like Android's /dev/binder. Its mutable type that implements the Property interface is RegularFileProperty. To get a RegularFile, turn to Directory#file(String). For fetching Provider<RegularFile>, leverage ProjectLayout#file(Provider<File>) or Directory#file(Provider<? extends CharSequence>).

RegularFile and Directory are two typed interfaces that provide improved clarity. The definition ensures that there are no ambiguities, with IDE hints and compiler checks ensuring the integrity of parameters' types.

However, FileCollection hasn't received a @Deprecated annotation, it remains suitable for specific use cases. For example, when there's no distinction between task inputs being files or directories, FileCollection is an ideal solution for accommodating a wide variety of files and folders:

val collection: FileCollection = project.files(
  "src/file1.txt",
  File("src/file2.txt"),
  listOf("src/file3.csv", "src/file4.csv"),
  Paths.get("src", "file5.txt")
)

Also, when preserving the hierarchy of several files or folders is crucial, turning to FileTree is the right move.

As we advance, let's discuss how we can transition between the Provider and FileSystemLocation combination and the FileCollection. Even though the majority of AGP tasks have transitioned to the Provider and FileSystemLocation approach, we often find artifacts in the core Gradle API where the use of FileCollection is indispensable. This necessitates a way to switch between these two types:

• Convert from Directory to FileCollection: You can use Directory.getAsFileCollection() or DirectoryProperty.getAsFileCollection().
• For a change from Directory to FileTree: Employ Directory.getAsFileTree() or DirectoryProperty.getAsFileTree().
• Transitioning from FileCollection to FileSystemLocation involves: FileCollection.getElements(), which yields Property<Set<FileSystemLocation>>.

If you're in the initial stages of crafting your Gradle Plugin, our recommendation is to lean towards using the Provider + FileSystemLocation combo. Not only does it align with the modern Gradle API, but it also smooths the data transmission when you're collaborating with the AGP ecosystem.

Understanding map(...) and flatMap(...)

The methods map(...) and flatMap(...) of the Provider class enable us to produce a new Provider Y based on the transformation of the existing Provider X. To get a clearer picture, let's inspect their full method signatures:

public interface Provider<T> {
  <S> Provider<S> map(Transformer<? extends S, ? super T> transformer);  
  <S> Provider<S> flatMap(Transformer<? extends Provider<? extends S>,
    ? super T> t);
}

public interface Transformer<OUT, IN> {
  OUT transform(IN varIn);
}

Taking a moment to check the first generic declarations of the two Transformer parameters, specifically the ? extends S and the Provider<? extends S>, reveals:

• The map(...) method expects the outcome of transform(IN varIn) to be a Direct Property type. This is evident as S is usually not a Provider.
• Conversely, flatMap(...) anticipates the outcome of transform(IN varIn) to be a Lazy Property type, encapsulated within a Provider.

Let's delve deeper into scenarios where these methods are applied to a TaskProvider labeled tp1:

• If map(...) is used on TaskProvider tp1 or properties annotated with @Output or @OutputFile within the Task, the resultant Provider<S> p1 retains tp1's dependency details.
• Regardless of the circumstances, the Provider<S> p2 returned by flatMap(...) will carry the dependencies of the property itself utilized during the transform(...) computation (especially if it's a Provider).

Such implicit task dependency processing is vital. In Section 3-3, for instance, we showcased the AGP's Variant API v2 utility, which relies heavily on this technique. However, the above explanation might seem slightly abstract. To truly grasp the distinction between these methods, let's consider a few hands-on examples:

abstract class ProviderTestPlugin : Plugin<Project> {
  override fun apply(project: Project) {

    val taskProviderA = project.tasks.register(
      "testProviderA",
      TaskA::class.java
    ) {
      initialParam = "Raw Param"
      metadata.set(project.layout
        .buildDirectory
        .file("outputs/logs/productA.txt"))
    }

    val taskProviderB = project.tasks.register(
      "testProviderB",
      TaskB::class.java
    ) {
      // You can also use:
      // metadata.set(taskProviderA.map { it.metadata.get() })
      metadata.set(taskProviderA.flatMap { it.metadata })
    }

    val taskProviderC = project.tasks.register(
      "testProviderC",
      TaskC::class.java
    ) {
      // These two approaches will produce different results here
      
      // Execute C with below code will see A->B->C
      // metadata.set(taskProviderB.map { it.metadata.get() })

      // While execute C with below code will see A->C
      metadata.set(taskProviderB.flatMap { it.metadata })
    }

    val taskProviderD = project.tasks.register(
      "testProviderD",
      TaskD::class.java
    ) {
      // A->D
      // param.set(taskProviderA.map { 
      //   "Inject ${it.initialParam} for TaskD!"})

      // Just D
      param.set(taskProviderA.flatMap { 
        project.provider { "Inject ${it.initialParam} for TaskD!" } })
    }

    val taskProviderE = project.tasks.register(
      "testProviderE",
      TaskE::class.java
    ) {
      param.set(taskProviderA.map { it.initialParam })
    }
  }

  abstract class TaskA : DefaultTask() {
    @get:Input
    var initialParam: String = ""

    @get:OutputFile
    abstract val metadata: RegularFileProperty

    @TaskAction
    fun write() {
      metadata.get().asFile.writeText("ProductA plus $initialParam")
      logger.lifecycle("TaskA writes something to the metadata file.")
    }
  }

  abstract class TaskB : DefaultTask() {
    @get:InputFile
    abstract val metadata: RegularFileProperty

    @TaskAction
    fun print() {
      logger.lifecycle("TaskB reads ${metadata.get().asFile.readText()}.")
    }
  }

  abstract class TaskC : DefaultTask() {
    @get:InputFile
    abstract val metadata: RegularFileProperty

    @TaskAction
    fun print() {
      logger.lifecycle("TaskC reads ${metadata.get().asFile.readText()}.")
    }
  }

  abstract class TaskD : DefaultTask() {
    @get:Input
    abstract val param: Property<String>

    @TaskAction
    fun print() {
      logger.lifecycle(param.get())
    }
  }

  abstract class TaskE : DefaultTask() {
    @get:Input
    abstract val param: Property<String>

    @Internal
    val selfIntro: Provider<String> = param.map { 
      "This is TaskE which consumes $it from A." }

    @TaskAction
    fun print() {
      logger.lifecycle(selfIntro.get())
    }
  }
}

Here, TaskA serves as the basic Task. It ingests an initialParam input string and produces a computed outcome for the file hosted by metadata.

TaskB accepts input in the form of a metadata file. In this task, two analogous methods are implemented using taskProviderA with map(...) and flatMap(...). Here's how they work: Using map(...), we're ensuring the caller is a TaskProvider or that the property in the transform(...) function is decorated with @OutputXxx, resulting in TaskA being executed before TaskB. On the other hand, using flatMap(...), TaskB also relies on TaskA since the Provider in the transform(...) function derives from TaskA's @OutputFile. Thus, for both methods, TaskA is executed prior to TaskB.

TaskC also integrates metadata file input. Two methods are based on taskProviderB with @get:InputFile abstract val metadata: RegularFileProperty. Intriguingly, they don't yield identical results. Using map(...), there's still a dependency on TaskB since its caller remains TaskProvider. Consequently, the execution order is A->B->C. However, with flatMap(...), no direct link to TaskB exists. This is because the referred metadata is dependent on TaskA's output, leading to the execution sequence A->C.

Shifting our gaze to TaskD, which is essentially a mutant of TaskC. With map(...), the execution order is A->D, while with flatMap(...), only TaskD is executed. Ponder over the underlying rationale for a moment.

TaskE offers a glimpse into another Provider-based transformation inside a Task. With a dependency chain delineated as param -> selfIntro, the execution order is A->E.

A key takeaway from these various scenarios is that the get() function is typically best reserved for invoke within Task Action. This ensures that computations are deferred until the Task Execution phase. It's worth noting that the Provider API is still undergoing tweaks. For instance, Gradle 6.6 introduced the zip operator for merging elements across two Providers, a functionality reminiscent of tools like Kotlin and RxJava operators. Meanwhile, the forUseAtConfigurationTime() method, introduced in Gradle 6.5, was swiftly deprecated. Any Gradle API calls made to retrieve environment variables during the Configuration phase have been fine-tuned for the Configuration Cache.

Also, an implicit task dependency trick can be seen in the use of the FileCollection type:

tasks.register<Zip>("packageFiles2") {
  from(processTemplatesTask)
}

Diving into the from method's internals, we see it wraps the processTemplatesTask task using project.files(), then it pulls the task's outputs, adding them to the collection and ensuring packageFiles2 depends on the given task.

4-2-3: Property Constraints

When pairing Property with Extension and Gradle Kotlin DSL, the assignment using the equal sign (=) is off the table until Gradle 8.1. Instead, you'll need to make a direct call to the Property<T>#set(T) API:

slackNotification1 {
  // can't simply use token = ...
  token.set(...)
}

However, if you're working with the Gradle Groovy DSL, the token = ... notation remains valid all the time. Do remember that the equal sign can't be skipped (as in token ...). This functionality is possible because Groovy inherently supports getters and setters for Property.

4-2-4: Hands-on Demo: Determining Inputs in the Execution Phase

Remember we kicked off this section with a conceptual scenario: crafting an advanced Plugin, tlack, grounded in the basic slack plugin. The objective is to atutogenerate the message field before dispatch. Now, it's time to bring that concept to fruition.

In the tlack Plugin, we want to autonomously curate a collection of Artifacts Metadata influenced by the App's packaged files. Subsequently, this data will be channeled into the message field of the slack Plugin for transmission. This hands-on session builds upon the Plugin tweaks covered in Sections 2-4, highlighting the upgrade to Variant API v2 and working with the Artifact API to acquire APK specifics. For a closer look at these advanced APIs, Chapter 3 houses the exemplary code.

Recall our discussion on Provider and Property? We had already altered properties within both SlackExtension and SlackTask, encapsulating them as lazy types via Provider or Property. Our current focus rests on formulating the TlackPlugin and CollectArtifactsMetaDataTask:

abstract class TlackPlugin : Plugin<Project> {

  private val slackNotificationPluginApplied = AtomicBoolean(false)

  override fun apply(project: Project) {
    project.afterEvaluate {
      check(slackNotificationPluginApplied.get()) {""}
    }

    project.plugins.withType<AppPlugin> {
      slackNotificationPluginApplied.set(true)

      // Diving into the primary logic here.
      val androidExtension = project.extensions.getByType(
        ApplicationAndroidComponentsExtension::class.java)
      androidExtension.onVariants { appVariant ->
        // 1. Setting up parameters
        val artifactsLoader = appVariant.artifacts
          .getBuiltArtifactsLoader()
        val apkDir = appVariant.artifacts.get(SingleArtifact.APK)

        // 2. Task Registration
        val taskName = "collectArtifactsMetaData" + 
          appVariant.name.capitalize(Locale.ENGLISH)
        val metadataName = "artifacts-metadata-${appVariant.name}.csv"
        val tlackTaskProvider = project.tasks.register(
          taskName, CollectArtifactsMetaDataTask::class.java
        ) {
          builtArtifactsLoader.set(artifactsLoader)
          apkFolder.set(apkDir)
          metadata.set(project.layout.buildDirectory
            .file("outputs/logs/$metadataName"))
        }

        // 3. Bridging the Output from the Slack Task to its Input
        val taskName = "assembleAndNotify" + 
          appVariant.name.capitalize(Locale.ENGLISH)
        val slackTaskProvider = project.tasks.named(taskName)
          as TaskProvider<SlackNotificationTask>
        slackTaskProvider.configure {
          message.set(tlackTaskProvider.map { 
            it.metadata.get().asFile.readText() 
          })
        }
      }
    }
  }

  abstract class CollectArtifactsMetaDataTask : DefaultTask() {
    //... (rest of the code remains unchanged)
  }
}

In essence, our approach for tlack mirrors that of the slack plugin. The pivotal step in auto-generating the message encompasses two phases:

1. Utilizing the Artifact API, we fetch the APK file. Within the TaskProvider of CollectArtifactsMetaDataTask, as the task executed, this is transformed into a metadata file. By virtue of the implicit dependency on apkFolder, this task naturally depends on the APK packaging task.
2. Subsequently, the map(...) function is employed to transform the metadata file's content, as output by CollectArtifactsMetaDataTask, into text. This text is then directed to the message field of SlackNotificationTask. Relying on the implicit dependency of metadata (tlackTaskProvider.map{...}), the slack tasks depends on the tlack tasks, thus rounding off the entire workflow.

4-2-5: In Summary

This chapter delved into the nuances of the lazy configuration containers, as touched upon in Section 3-3. The goal is to pave a smoother path for developers to tailor the properties of Gradle Extension and Task. Of course, Property has more under its belt than what's been discussed – like restricting future value modifications, disAllowChanges() and setDisallowChanges() APIs are worth a look.