4-6: Plugin Testing

In the concluding section of this chapter, we will be refining the Slack notification plugin by adding the corresponding test code. In real-world project development, tests typically evolve alongside the main codebase, sometimes even preceding it, as seen in the Test-Driven Development (TDD) approach.

The source code used in this section can be found within the slack project:

• The Plugin project resides at slack/plugins/slack-test.
• Tinker with the Plugin configuration at slack/plugin-config/src/main/kotlin/slack-test.gradle.kts.
• The Android Application test project, a bearer of the Plugin, resides in slack/app. Remember to activate the appropriate Plugin in app.gradle.kts during testing.

4-6-1: Test Classification

Testing a Gradle Plugin is a bit similar to testing a regular JVM project and can be categorized into three segments:

• Unit Test: These are of the finest granularity, narrowed down to function-level. They do not interact with external systems and use mock tools to simulate external dependencies. The primary focus is usually within a class or file and is generally considered white-box testing.
• Integration Test: At medium granularity, they comprise multiple classes or files and might depend on external dependencies. The focus is typically on a module (referring to a Gradle Module).
• Functional Test (or System Test, End-to-End Test): These have a global perspective, encompassing the complete project environment (across multiple modules) or the final deliverable product. They depend on the actual runtime environment, with the focal point being the final functional output, and are more black-box in nature.

In theory, the classic test pyramid (Figure 4.6.1) suggests that there should be a gradual reduction from Unit to Integration to Functional tests. Any code that can be covered in unit tests should not be left for subsequent stages. However, decisions should be based on the current project's specifics (as discussed further in the Gradle Test Suites portion), some people might merge the concepts of functional testing and integration testing, which we will not delve into further. This book solely discusses the three distinct categories mentioned above.

4-6-2: Test Doubles

In the three test categories, the concept of "external dependencies" is often referred to, either simulated or real. In practical application scenarios, we commonly rely on the following three environments:

For instance, considering the "interfaces and implementations" scenario:

• Mock: An implementation of one or multiple interfaces with null internal logic. It simulates the return values of various APIs and can be dynamically modified. Generally, it's based on certain Mock frameworks.
• Fake: Similar to Mock but cannot be dynamically changed and might just have a simple version of the actual implementation.
• Real: The genuine implementation of an interface, containing all the complete logic.

Both Mocks and Fakes are considered ways to simulate the real environment, commonly known as Test Doubles. There are more specific variants of these, such as Stubs (which return null results and are used to check if methods were called) and Spies (mock only a portion of a real object's API), which won't be detailed here.

4-6-3: Gradle Test Suites

For the three types of tests (unit, integration, and functional), there are multiple corresponding test methods and accompanying tools.

Unit testing of a Gradle Plugin doesn't have specific requirements and is not much different from a typical JVM. Due to Gradle's historical reliance on Groovy, the official documentation recommends Spock as the Runner. For our samples in this section, we use the fundamental JUnit as the Runner, with Hamcrest complementing the Matcher section. Additionally, to facilitate mocking Kotlin objects, the Mockk framework is incorporated. These tools are used not just for unit tests but also for the other two test types.

For integration testing, we mainly employ the org.gradle.testfixtures.ProjectBuilder helper class to programmatically create a pristine project. It provides flexibility, allowing the dynamic addition of Plugin, Extension, Task, Shared Build Service, etc.

val project = ProjectBuilder.builder()
  .withName("extension-test")
  .build()

val slackExt = project.extensions.create(
  "slackNotification",
  SlackNotificationExtension::class.java
).apply {
  enabled.set(true)
  defaultConfig {
    message.set(msg)
    pkg {
      id.set(pkgId)
    }
  }
  channels.apply {
    register("androidTeam") {
      token.set(slackToken)
      channelId.set(slackChannelId)
    }
    register("androidTeam2") {
      token.set(slackToken)
      channelId.set(slackChannelId)
    }
  }

  selectChannelsByVariant { variant, channel ->
    !(variant == "debug" && channel == "mobileTeam")
  }
}

assertThat("...", slackExt.enabled.get())
assertThat("...", slackExt.defaultConfig.message.get() == msg)
assertThat("...", slackExt.defaultConfig.pkg.id.get() == pkgId)
assertThat("...", slackExt.channels.toList().size == 2)

This code snippet creates a project named extension-test, which is accessible from the current context. It then uses it to add a SlackNotificationExtension to test if complex DSL rules are successfully configured and executed.

For functional testing, Gradle offers a set of tools called Gradle Test Suites with org.gradle.testkit.runner.GradleRunner serves as an entry point. It's based on the Gradle Tooling API, providing insights into various states during a real Gradle project compilation, such as Task execution status and post-compilation results.

val expectFailure = true // Or false
val runner = GradleRunner.create()
  .withPluginClasspath()
  .withProjectDir(sampleAppDir)
  .withTestKitDir(testDir)
  .withArguments("clean", "assembleDebug")
val result:BuildResult = if (expectFailure) {
  runner.buildAndFail() 
} else {
  runner.build()
}
// result.task(taskPath)?.outcome != TaskOutcome.FAILED

Let's break it down:

1. Makes use of withPluginClasspath() to add the current module's Plugin and corresponding source set to the Compile Classpath of the project being tested. This allows us to directly load the Plugin without having to publish it to a remote or local repository. Here, the term "Plugin" refers to the settings related to gradlePlugin, such as the settings for slack-test project (as shown below). By default, only the test codes within the test type Source Set that execute this method will successfully link the project being tested with the Plugin's classpath. If you have other test types (e.g., integrationTest), they need to be added manually.

   gradlePlugin {
     plugins.register("slack-test") {
       id = "me.2bab.buildinaction.slack-test"
       implementationClass = "me.xx2bab.buildinaction.slacktest." + 
         "SlackNotificationPlugin"
     }
     testSourceSets.add(sourceSets["integrationTest"])
     testSourceSets.add(sourceSets["functionalTest"])
   }

2. withProjectDir(String s) specifies the directory of the project to test.

3. withTestKitDir(String s) defines an internal directory used by GradleRunner. Unless there are specific requirements, it doesn't need to be specified. During the execution of the test project, it serves as the Gradle Home, local cache directory, etc., for the project being tested.

4. withArguments(String...args) denotes the commands and parameters the test project will execute.

5. runner.build() and runner.buildAndFail() trigger the build, reflect the expected outcome of the test, functioning as basic assert commands in functional testing.

For more environment parameters, like setting the Gradle version, refer to the GradleRunner API documentation.

In the testing workflow, using GradleRunner is pivotal, especially when many Plugin types are not amenable to unit testing, for instance, when a Task is extended from the AGP to produce additional files. In practice, the functional test coverage of many open-source Gradle Plugins often surpasses that of unit tests. Software engineering is about making trade-offs, and there's no one-size-fits-all solution. If functional testing is more suitable for the Gradle Plugin context, then we shouldn't be overly concerned about whether our test coverage resembles a "pyramid" or an "ice cream cone".

Additionally, the Test Fixture , typically found within the fixture directory of a Gradle Plugin project, refers to one or multiple simplified test projects. They are designed to encompass the variety of conditions under which Plugins would run in a real environment. This concept is a staple in the testing realm. For example, AGP 7.x introduced similar functionality for Android testing:

android {
  testFixtures {
    enable = true
  }
}

In Plugin testing, constructing the environment can either involve static project files or dynamically created and written files during test execution. Usually, it's a blend of both: creating one or two project directories containing source code, resources, etc., while dynamically writing the build.gradle(.kts) to adjust different Plugin configurations (i.e., modifying the Plugin DSL settings).

4-6-4: Emphasis in Testing Various Gradle Plugin Types

From a standpoint of ease in testing and reusability (isolating platform APIs), when designing a Plugin, it's conventional to segment the project into:

1. Components interacting with the Gradle API, its ecosystem (like AGP), and platform specifics – focusing mainly on templated logic. This is referred to as the plugin-entry portion in the following content. The crux of its testing revolves around integration and functional tests, such as executing $gradlew clean assemble in a real Application's build environment.
2. The core business logic, dubbed the plugin-core , where plugin-entry depends on plugin-core. The emphasis for testing the business logic lies in unit and integration tests, for instance:
   1. Running unit tests on each well-decoupled class.
   2. Performing integration tests on a combination of partial capabilities or the entirety of the plugin-core module.

Given the aforementioned segmentation, taking AGP as an example, core functionalities like AAPT2, ManifestMerge, Kotlin Compiler, etc., are independent subsystems. Some, like AAPT2, aren't even JVM-based (AAPT2 is written in C++ and produces standalone executables). AGP, serving as a caller module, invokes such executables via the ProcessBuilder API. This modular approach facilitates more isolated and straightforward unit and integration testing.

While AGP exemplifies the Ecosystem Plugin, considerations for the Gradle-Ext Plugin and Ecosystem Synergy Plugin differ slightly:

• Gradle-Ext Plugins, being dependent solely on the Gradle API and having fewer external dependencies, often involve creating different build.gradle(.kts) configurations and injecting them into the Fixture project during the testing phase. Their adaptability necessitates compatibility with diverse ecosystems and various Gradle versions.
• Ecosystem Synergy Plugins, because they add dependencies related to ecosystem-specific APIs, generally have more fixed project configuration templates. During testing, one might opt to make minimal changes to the build.gradle(.kts) in the Fixture project environment, focusing on altering configurations of the Plugin extension. Due to their tight linkage with the evolution of the ecosystem, the primary adaptation focus is on catering to multiple versions of that ecosystem itself.

4-6-5: Hands-on Demo: Incorporating Test Code for Slack Plugins

The complexity of software engineering tests extends far beyond the cursory introduction above, which only scratched the surface of Gradle Plugin testing. We'll delve deeper into actual test configurations and writing strategies using a demo.

The accompanying project for this section, the slack-test Plugin, is essentially an enhancement of the slack-cache-rules-compliance Plugin. It's an endeavor to fine-tune the Plugin, ensuring it's more decoupled and testing-friendly. For details on its core functionalities, please refer to earlier sections of Chapter 4. Here, our focus is on its optimization and testing.

Refactoring

Revisiting the structure of the project slack-cache-rules-compliance:

├── slack-cache-rules-compliance
│  └── src/main/kotlin                 
│    └── me.xx2bab.buildinaction.slackcache
│      ├── DefaultConfig.kt
│      ├── DummyHttpClient.kt
│      ├── Package.kt
│      ├── PrivateCloudService.kt
│      ├── SlackChannel.kt
│      ├── SlackNotificationExtension.kt
│      ├── SlackNotificationPlugin.kt
│      ├── SlackNotificationTask.kt
│      └── SourceToVerificationCodesTask.kt
│  ├── build.gradle.kts
│  └── settings.gradle.kts

Informed by our earlier discussion on "Emphasis in Testing Various Gradle Plugin Types", we can enumerate some actionable strategies to enhance the project's architecture and class organization:

1. Isolate platform-agnostic core logic, data repositories, etc., and place them in the core package.
2. Separate logic-free model classes (typically data class or Gradle Extension) and lodge them within the model package.
3. For non-Gradle-Managed Classes/Properties, can still consider Dependency Injection (DI) to integrate external dependencies. For this purpose, we've crafted an Initializer.kt to facilitate manual Dependency Injection.

├── slack-cache-rules-compliance               
│   └── src
│     └── main/kotlin
│       └── me.xx2bab.buildinaction.slackcache
│         ├── PrivateCloudService.kt       
│         ├── SlackClientService.kt        
│         ├── SlackNotificationExtension.kt
│         ├── SlackNotificationPlugin.kt
│         ├── SlackNotificationTask.kt
│         ├── SourceToVerificationCodesTask.kt
│         ├── core
│           ├── DummyHttpClient.kt
│           ├── HashCalculator.kt
│           ├── Initializer.kt
│           ├── SlackAPI.kt
│           ├── SlackClient.kt
│         ├── model
│           ├── ...
│   ├── build.gradle.kts
│   └── settings.gradle.kts

Contrary to the belief that we refactor solely for the sake of testing, the reality is quite the opposite. Code that is easily testable tends to be the code where individual components can be smoothly changed or replaced—a testament to high cohesion and low coupling. Refactoring for testability primarily revolves around "maintainability." Whether we aim for peace of mind during software releases or ease in extending or replacing certain components in the future, our endgame is predictability.

Before we delve into the specifics, let's first examine the structural layout of the test code relative to the original source code structure.

├── slack-cache-rules-compliance               
│   └── src
│     └── functionalTest/kotlin
│       └── me.xx2bab.buildinaction.slackcache
│         ├── ProjectIntegrityTest.kt
│         ├── SlackNotificationTaskTest.kt
│         └── SourceToVerificationCodesTaskTest.kt
│     ├── integrationTest/kotlin
│       └── me.xx2bab.buildinaction.slackcache
│         ├── PrivateCloudServiceTest.kt
│         ├── SlackClientServiceTest.kt
│         └── SlackNotificationExtensionTest.kt
│     ├── test/kotlin
│       └── me.xx2bab.buildinaction.slackcache
│         ├── DummyHttpClientTest.kt
│         ├── HashCalculatorTest.kt
│         └── SlackClientTest.kt
│     └── main/kotlin
│       └── ...
│   ├── build.gradle.kts
│   └── settings.gradle.kts

Setting Up the Testing Environment

Gradle's documentation on Plugin testing, as of version 7.4.2, suggests manual configuration of Configuration, SourceSet, and other related environments. However, starting with Gradle 7.3, a new Test Suites API was introduced (not to be confused with TestKit). This API makes setting up a testing environment more efficient and intuitive. Let's examine the setup process for the slack-test Plugin, using the Test Suites API as a model:

val deleteOldInstrumentedTests by tasks.registering(Delete::class) {
  delete(layout.buildDirectory.dir("test-samples"))
}

testing {  // [I]
  suites {
    val test by getting(JvmTestSuite::class) {
      useJUnitJupiter()
    }

    val integrationTest by registering(JvmTestSuite::class) {
      dependencies {
        implementation(project)
        implementation(project(":gradle-instrumented-kit"))
      }
      targets {
        all {
          testTask.configure {
            shouldRunAfter(test)
            dependsOn(deleteOldInstrumentedTests)  // [VI]
          }
        }
      }
    }

    val functionalTest by registering(JvmTestSuite::class) {
      dependencies {
        implementation(project)
        implementation(project(":gradle-instrumented-kit"))  // [VI]
      }
      targets {
        all {
          testTask.configure {
            shouldRunAfter(test)
            dependsOn(deleteOldInstrumentedTests)
          }
        }
      }
    }
  }
}

configurations["integrationTestImplementation"]
  .extendsFrom(configurations["testImplementation"])  // [II]
configurations["functionalTestImplementation"]
  .extendsFrom(configurations["testImplementation"])

tasks.check.configure {  // [III]
  dependsOn(tasks.named("test"))
  dependsOn(tasks.named("integrationTest"))
  dependsOn(tasks.named("functionalTest"))
}

tasks.withType<Test> {
  testLogging {
    this.showStandardStreams = true  // [IV]
  }
}

dependencies {
  ...
  // Tests
  testImplementation(deps.hamcrest)
  testImplementation(deps.mockk)
  testImplementation(gradleTestKit())
}

gradlePlugin {
  testSourceSets.add(sourceSets["integrationTest"])  // [V]
  testSourceSets.add(sourceSets["functionalTest"]) 
}

• [I] The testing {...} DSL configuration represents the main body of the Test Suites. It requires the inclusion of Gradle's java Plugin. Generally, there's no need to specifically apply the java plugin as Plugins like Android, Kotlin, or java-gradle-plugin reference the java Plugin inside.

• Inside suites{...}, you can either obtain or register the desired test type using delegation like val xxx by getting(JvmTestSuite::class) or registering(JvmTestSuite::class). The test type is predefined, so you use getting, while other custom types require registering.

• [II] Since the current version of Test Suites does not support accessing some of Gradle's predefined dependencies in JvmTestSuite#dependencies, like gradleTestKit(), it's advisable to have both integrationTest and functionalTest extend from test. This ensures the dependency can be incorporated into all types of test projects with just a single testImplementation(gradleTestKit()) line.

• [III] The tasks.check serves as an Lifecycle Task to anchor our custom test Tasks, it can be thought of as an assemble used typically during packaging.

• [IV] This commonly-used technique retrieves all test Tasks and enables their log output for easy observation, especially useful for examining the compilation logs from the GradleRunner during functionalTest.

• [V] Lastly, both functionalTest and integrationTest rely on a custom-made instrumented-kit dependency for shared utility tools. Generally, if there's no integrationTest, the auxiliary toolkit might be created directly in the functional test directory. However, when both exist, it's worth considering breaking out this environment aid into a separate module. Here's how its structure relates to slack-test:

  ├── gradle-instrumented-kit
  │   └── src/main/kotlin
  │     └── me.xx2bab.buildinaction.slacktest.base
  │         ├── DepVersions.kt
  │         ├── EnvProperties.kt
  │         ├── GradleRunnerExecutor.kt
  │         └── SlackConfigs.kt
  └── slack-test

For the slack-test project, the utility tools serve the following purposes:

1. They wrap the GradleRunner, enabling external test codes to execute relevant compile commands with a single line, simplifying the testing process.
2. They offer both the project's and the system's environmental variables. For instance, when injecting different AGP versions dynamically, one can read from the predefined AGP values in the external project's Version Catalog and subsequently input them.
3. They provide DSL configuration script snippets for the Plugin under test. External test codes can integrate various configurations into a single line, further streamlining the testing process.

Developers may also want to draw inspiration from the following or consider crafting more reusable utility sets themselves. In this context, we'll delve into the encapsulation logic of the GradleRunnerExecutor, including a discussion of several GradleRunner APIs.

object GradleRunnerExecutor {

  val versions = DepVersions()
  val envProps = EnvProperties()

  private fun initTestResources(): File {
    val sampleAppDir = File("build/test-samples/", 
      "test-sample-${System.currentTimeMillis()}" + 
      "-${Random.nextInt(1024, 2048)}")
    File("src/fixtures/test-sample").copyRecursively(sampleAppDir)
    return sampleAppDir
  }

  // For public invocation
  fun execute(config: String, vararg tasks: String) 
    = execute(config, false, *tasks)

  // For public invocation
  fun executeExpectingFailure(config: String, vararg tasks: String)
    = execute(config, true, *tasks)

  private fun execute(
    config: String,
    expectFailure: Boolean,
    vararg tasks: String,
  ): ExecuteResult {
    val sampleAppDir = initTestResources()

    File(sampleAppDir, "settings.gradle.kts").appendText(
      """
      pluginManagement {
        plugins {
          // AGP Application Plugin
          id(...) version "${versions.getVersion("agpVer")}" apply false
          // Test Plugin
          id("me.2bab.buildinaction.slack-test") version "1.0" apply false
        }
        repositories {
          gradlePluginPortal()
          google() 
          mavenCentral()
        }
      }
    """
    )

    File(sampleAppDir, "app/build.gradle.kts").appendText(
      """
      $config
    """
    )

    return executeGradle(sampleAppDir, expectFailure) {
      withArguments(*tasks)
    }
  }

  private fun executeGradle(
    sampleAppDir: File,
    expectFailure: Boolean,
    block: GradleRunner.() -> Unit,
  ): ExecuteResult = runWithTestDir { testDir -> // [I]
    val runner = GradleRunner.create()
      .withPluginClasspath()
      .withProjectDir(sampleAppDir)
      .withTestKitDir(testDir)
      .apply(block)

    if (!expectFailure) {
      runner.withArguments(runner.arguments.toMutableList()
        + "-s") // Add stacktrace
    }

    val result: BuildResult = if (expectFailure) {
      runner.buildAndFail() 
    } else {
      runner.build()
    }

    if ("--debug" !in runner.arguments) {
      println(result.output)
    }

    ExecuteResult(
      sampleAppDir,
      result
    )
  }
  ...
}

The two key APIs exposed by GradleRunnerExecutor are execute(...) and executeExpectingFailure(...). They correspond to runner.build() and runner.buildAndFail() respectively. To run a test project, it is essential to copy the fixture project to a target directory to obtain a clean testing environment. This setup also dynamically writes some test scripts based on the current external environment variables. Yet, it leaves room for external callers to utilize some configurations, such as the Plugin's extension configuration in build.gradle.kts. Lastly, this encapsulation includes some debugging parameters and final result outputs.

When running the tests, we've wrapped a function (at [I]), runWithTestDir(...). This approach is inspired by the optimization strategy from the gradle-play-publisher (https://github.com/Triple-T/gradle-play-publisher) Plugin's tests. Every Gradle Runner starts its Daemon, and when multiple tests run concurrently without specifying withTestKitDir(...), CPU resource shortages coupled with frequent file lock contention can degrade performance. However, designating a directory for every test isn't the optimal solution either. Here, we've adopted a thread pool approach, configuring a certain number of test directories based on the number of cores and queuing Tasks sequentially.

private val testDirPool: BlockingQueue<File>

init {
  val random = Random(System.getProperty("user.name").hashCode())
  val tempDir = System.getProperty("java.io.tmpdir")

  val threads = max(1, Runtime.getRuntime().availableProcessors() / 4 - 1)
  val dirs = mutableListOf<File>()
  repeat(threads) {
    dirs.add(File("$tempDir/gppGradleTests${random.nextInt()}"))
  }

  testDirPool = ArrayBlockingQueue(threads, false, dirs)
}

private fun <T> runWithTestDir(block: (File) -> T): T {
  val dir = testDirPool.take()
  try {
    return block(dir)
  } finally {
    testDirPool.put(dir)
  }
}

Unit Testing

For unit testing, after splitting the core code according to the ideas discussed earlier (and excluding platform dependencies), it's just like standard JVM unit testing. Let's take the HashCalculator class test as an example to understand a streamlined testing process:

class HashCalculatorTest {

  private val hashCalculator = HashCalculator()

  @Test
  fun md5_successful() {
    val res = hashCalculator.md5("abcdefghijklmn")
    assertThat(
      "Issue with MD5 calculation...",
      res == "0845a5972cd9ad4a46bad66f1253581f"
    )
  }

  @Test
  fun sha256_successful() {
    val res = hashCalculator.sha256("abcdefghijklmn")
    assertThat(
      "Issue with MD5 calculation...",
      res == "0653c7e992d7aad40..."
    )
  }
}

Integration Testing

Earlier, we briefly discussed the usage of ProjectBuilder and tested the SlackNotificationExtension. Let's delve deeper with an example of a Shared Service:

class SlackClientServiceTest {

  @Test
  fun `SlackClientService posts successfully`() {
    val envProperties = EnvProperties()
    val slackToken = envProperties.slackToken
    val slackChannelId = envProperties.slackChannelId
    val msg = "built successfully"

    val project = ProjectBuilder.builder()
      .withName("slack-client-test")
      .build()

    val slackClientService = project.gradle.sharedServices
      .registerIfAbsent("...", SlackClientService::class.java) {}
    val succResp = slackClientService.get()
      .postNewMessage(slackToken, slackChannelId, msg)
    assertThat("Error encountered while posting: ${succResp.error}",
      succResp.ok)
  }
  
  ...
  
}

We simply checked the postNewMessage(...) function from the SlackClientService in the above code snippet. While ProjectBuilder can specify more parameters such as ProjectDir and GradleHome and is apt for testing Tasks, Plugins, and so on, I personally believe that unless it's a Gradle-Ext Plugin, it's challenging to test within this minimalistic Project environment. Most Tasks and Plugins are better suited for the functional testing environment provided by GradleRunner.

Functional Testing

For functional testing, we first used a clean Task execution to verify the project's configuration integrity. This was followed by a assembleDebug Task execution check. This essentially aligns with Gradle's two lifecycles.

class ProjectIntegrityTest {

  @Test
  fun `Check Gradle scripts integrity`() {
    val taskPath = ":app:clean"
    val result = execute(
      SlackConfigs.regular(envProps.slackToken, 
        envProps.slackChannelId),
      taskPath
    )
    assertThat(
      "...compilation errors...leading to failure of the clean task.",
      result.buildResult.task(taskPath)?.outcome != TaskOutcome.FAILED
    )
  }

  @Test
  fun `Check project integrity by assembling`() {
    val taskPath = ":app:assembleDebug"
    val result = execute(
      SlackConfigs.regular(envProps.slackToken, envProps.slackChannelId),
      taskPath
    )
    assertThat(
      "...compilation errors...causing the assemble task to fail.",
      result.buildResult.task(taskPath)?.outcome != TaskOutcome.FAILED
    )
  }

}

Let's also evaluate the specific Plugin Task testing:

class SlackNotificationTaskTest {

  @Test
  fun `Assemble and Notify the Slack Bot successfully`() {
    val taskPath = ":app:assembleAndNotifyDebug"
    val result = execute(
      SlackConfigs.regular(envProps.slackToken, 
        envProps.slackChannelId),
      taskPath
    )
    assertThat(
      "...",
      result.buildResult.task(taskPath)?.outcome != TaskOutcome.FAILED
    )
    val csvFile = File(result.projectDir, 
      "app/build/outputs/logs/slack-notification.csv")
    val csvData: List<Map<String, String>> 
      = csvReader().readAllWithHeader(csvFile.readText())
    for (data in csvData) {
      assertThat(
        "The request of ${data["requestOf"]} was not successful.",
        data["responseCode"]!!.toBoolean()
      )
    }
  }

  @Test
  fun `Build fails when SlackNotification is not configured`() {
    val taskPath = ":app:assembleAndNotifyDebug"
    val result = executeExpectingFailure(
      SlackConfigs.partialConfiguratedOnly,
      taskPath
    )
    assertThat(
      "Extension configuration was incomplete," +
      " yet the build unexpectedly succeeded.",
      result.buildResult.task(taskPath) == null
    )
  }

}

Here, we test both successful and failure scenarios using two different SlackNotificationExtension configurations. Their source code is as follows:

object SlackConfigs {

  fun regular(token: String, channel: String) = """
    slackNotification5 {
      enabled.set(true)
      defaultConfig {
        message.set("...")
        pkg {
          id.set("id")
        }
      }
      channels {
        register("androidTeam") {
          token.set("$token")
          channelId.set("$channel")
        }
        register("androidTeam2") {
          token.set("$token")
          channelId.set("$channel")
        }
      }
    
      selectChannelsByVariant { variant, channel ->
        !(variant == "debug" && channel == "mobileTeam")
      }
    }
  """

  val partialConfiguratedOnly = """
    slackNotification5 {
      enabled.set(true)
    }
  """.trimIndent()
  
}

When assessing the Task's execution result with result.buildResult.task(taskPath)?.outcome != TaskOutcome.FAILED, it's possible to utilize the inverse condition, conveniently covering SUCCESS, UP_TO_DATE, FROM_CACHE, and so on. However, if your Task doesn't allow skipping statuses like SKIPPED or NO_SOURCE, or if you aim to test cache execution, you should explicitly specify the test result and broaden the scenario coverage.

To enhance the testability of this Task, we've introduced an additional csv file output. This helps in determining the specific outputs of a Task, and a standardized output structure also facilitates third-party Tasks or Plugins in extending our Task scenarios.

4-6-6: In Summary

This section on Gradle Plugin testing merely scratches the surface. Due to space constraints, some classes haven't been detailed here, but they can be found in the project source code. Through this section, I hope to provide readers with fresh insights into Gradle Plugin testing, guiding them to discern the most value-added testing strategies suitable for their teams.