3-4: AGP Configuration Procedure

In this section, we'll be setting out on an exciting journey to uncover the configuration procedure of the Android Gradle Plugin (AGP). This will include the discovery of entry points and the intricate process of building Variants and Tasks. Along the way, I'll be sharing the thought process behind my approach to "identifying issues - analyzing the source code - finding solutions" when crafting AGP Synergy Plugins.

3-4-1: The Route from "Identifying Issues - Analyzing the Source Code - Finding Solutions"

When tackling source code,it's crucial to narrow in on specifics; otherwise, it's easy to become overwhelmed by the sheer volume of code and have no idea where to begin. Since we are not AGP's creators, we are primarily concerned with optimizing our interactions with it. To make things engaging, I've curated three small problems I've encountered in the past. We'll unravel the answers to these as we proceed with our analysis:

1. What caused the internal logic of AppPlugin to be hollowed out when upgrading from AGP 3.5 to 3.6 and subsequent versions? A cursory glance at the source code reveals that AppPlugin is actually proxied, but why is that so?
2. What resources and services are available to AppPlugin or BasePlugin? How do we discover them and acquire instances? For example, how do we fetch the relevant paths for the current version of Android SDK and CLI tools?
3. If we wish to perform a unique operation on the Variant object right after it's created and before all the onVariants(...) callbacks are triggered, how do we step in?

We've already dipped our toes into the concepts and functions of AGP in Section 1-2, and the techniques for AGP source code debugging were shared in Section 2-7. The aim of this section is to enable you to find debugging entry points swiftly when faced with a problem, derive insights from it, and find solutions or compatibility fixes. To achieve this, we need to understand AGP's configuration processes, Task creation, and other crucial takeaways to address the questions above. Later, in the next section, we'll complete the thought process of "problem - source code analysis - solution" via an intricate case study (creation and usage of Artifact).

However, we're not going to deep-dive into the architecture of AGP by meticulously combing through the whole source code. Firstly, AGP evolves rapidly, and it's not rare to see a year-old design revamped in a year or two, without any disclosed architectural design guide from the officials. Secondly, giving a mere list of Tasks' traversal within AGP won't really enhance your skills. It would be far more helpful to teach you to fish by changing the way you think.

3-4-2: Entry Point of the AppPlugin

Now, let's get back on track. If you've ever written an Android application, you've undoubtedly come across the com.android.application plugin. The module that applies it becomes the primary module of an Android Application. In the upcoming sections, we'll be using the com.android.application plugin as our guide to explore its entry points and configuration process. You can apply the same analysis to the rest of the AGP plugins as well.

From Section 2-4, we know that the Gradle plugin's declaration resides in the META-INF/gradle-plugins directory of the jar package. Let's take com.android.application as an example. Directly opening this directory and locating the declaration file com.android.application.properties corresponding to the ID, reveals that it points to:

implementation-class=com.android.build.gradle.AppPlugin

Checking out the code for AppPlugin might leave you scratching your head—why does it have just one line of code calling apply(INTERNAL_PLUGIN_ID)?

class AppPlugin: BasePlugin() {
  override fun apply(project: Project) {
    super.apply(project)
    project.apply(INTERNAL_PLUGIN_ID)
  }
}

private val INTERNAL_PLUGIN_ID = mapOf(
  "plugin" to "com.android.internal.application")

If we search for the implementation of com.android.internal.application, we can find that it points to com.android.build.gradle.internal.plugins.AppPlugin.

public class AppPlugin extends AbstractAppPlugin<
        com.android.build.api.dsl.ApplicationExtension,
        ApplicationAndroidComponentsExtension,
        ApplicationVariantBuilderImpl,
        ApplicationVariantImpl> {
  @Inject
  public AppPlugin(
      ToolingModelBuilderRegistry registry,
      SoftwareComponentFactory componentFactory,
      BuildEventsListenerRegistry listenerRegistry) {
    super(registry, componentFactory, listenerRegistry);
  }


  @NonNull
  @Override
  protected ExtensionData<ApplicationExtension> createExtension(
      @NonNull DslServices dslServices,
      @NonNull
          DslContainerProvider<...>
              dslContainers,
      @NonNull NamedDomainObjectContainer<BaseVariantOutput> buildOutputs,
      @NonNull ExtraModelInfo extraModelInfo,
      @NonNull VersionedSdkLoaderService versionedSdkLoaderService) {
    ...
  }

  @NonNull
  @Override
  .. ApplicationAndroidComponentsExtension createComponentExtension(..) {
    SdkComponents sdkComponents =
        dslServices.newInstance(
            SdkComponentsImpl.class,
            dslServices,
            project.provider(getExtension()::getCompileSdkVersion),
            project.provider(getExtension()::getBuildToolsRevision),
            project.provider(getExtension()::getNdkVersion),
            project.provider(getExtension()::getNdkPath),
            project.provider(bootClasspathConfig::getBootClasspath));

    ApplicationAndroidComponentsExtension extension =
        project.getExtensions()
            .create(
                ApplicationAndroidComponentsExtension.class,
                "androidComponents",
                ApplicationAndroidComponentsExtensionImplCompat.class,
                dslServices,
                sdkComponents,
                variantApiOperationsRegistrar,
                getExtension());

    return extension;
  }

  @NonNull
  @Override
  protected ApplicationTaskManager createTaskManager(...) {
    return new ApplicationTaskManager(
        project,
        variants,
        testComponents,
        testFixturesComponents,
        globalTaskCreationConfig,
        localConfig,
        extension);
  }

  ...
}

The AppPlugin under internal.plugins is the true entry point we've been seeking, and the above code is its simplified class file. In Section 3-3, we discussed AGP's new packaging strategy starting from version 4.1, and this double-layered AppPlugin is a result of transformations starting from AGP version 3.6.

By examining the git commit information for the associated classes, we discover that the original single-layer AppPlugin mechanism was inadequate to meet the demands of the future new package strategy. Given that external users require Plugin references to apply, the Plugin needed to be located in the :gradle-api package for future reference. However, the AppPlugin implementation doesn't expose any APIs directly to users, and these finer details needed to be kept away in the non-public :gradle package. Thus, a Plugin proxy approach was born, allowing the inner layer AppPlugin to iterate freely and safely. This answers our first preset question: if you wish to access the associated attributes of AppPlugin, you need to switch to AppPlugin under internal.plugins in the new version.

Starting with this version, AGP's source code began to feature Kotlin as its primary programming language, leading to a mixed development environment with existing Java classes.

The inner AppPlugin primarily implements some methods from the parent classes AbstractAppPlugin and BasePlugin, and determined application-specific generic parameters like ApplicationExtension, ApplicationAndroidComponentsExtension, ApplicationVariantImpl. However, the main apply(...) logic of the Plugin is still traced back to BasePlugin.

@Override
public final void apply(@NonNull Project project) {
  CrashReporting.runAction(
    () -> {
    basePluginApply(project);
    pluginSpecificApply(project);
    project.getPluginManager().apply(AndroidBasePlugin.class);
  });
}

The AndroidBasePlugin is an empty Plugin implementation that doesn't extend any other base class. Its main purpose is to allow other Plugin developers to ascertain whether the current Project environment is related to Android (not limited to Application or Library)—this can be detected using the project.plugins.findPlugin(...) method. The pluginSpecificApply(...) method needs to be implemented by subclasses, and the one in AppPlugin is actually empty, so we can skip it. The basePluginApply(...) method involves a significant amount of code, and when reviewing the major set-up flow, you can temporarily bypass the following types of logic or specific implementation classes:

• Tool classes by BuildService, general utility classes, or data collection types, often reused in subsequent configuration processes. Refer to Section 4-4 for more details on BuildService.
• Data statistic and crash collection auxiliary services, usually used to wrap the execution of key steps.
• Environmental safety checks, including JVM, dependencies, Project parameters, SDK version, etc.

The primary steps in basePluginApply(...) are actually calling the configureProject, configureExtension, and createTasks methods.

private void basePluginApply(@NonNull Project project) {
  System.setProperty("java.awt.headless", "true");
  this.project = project;
  new AndroidLocationsBuildService.RegistrationAction(project)
    .execute();
  optionService = new ProjectOptionService.RegistrationAction(project)
    .execute().get();
  createProjectServices(project);
  ...
  
  GradleBuildProject.Builder projectBuilder =
  configuratorService.getProjectBuilder(project.getPath());
  if (projectBuilder != null) {
    projectBuilder
      .setAndroidPluginVersion(Version.ANDROID_GRADLE_PLUGIN_VERSION)
      .setAndroidPlugin(getAnalyticsPluginType())
      .setPluginGeneration(GradleBuildProject.PluginGeneration.FIRST)
      .setOptions(AnalyticsUtil.toProto(projectOptions));
  }

  // Three key methods will be called by a record service
  configuratorService.recordBlock(
    ExecutionType.BASE_PLUGIN_PROJECT_CONFIGURE,
    project.getPath(),
    null,
    this::configureProject);

  configuratorService.recordBlock(
    ExecutionType.BASE_PLUGIN_PROJECT_BASE_EXTENSION_CREATION,
    project.getPath(),
    null,
    this::configureExtension);

  configuratorService.recordBlock(
    ExecutionType.BASE_PLUGIN_PROJECT_TASKS_CREATION,
    project.getPath(),
    null,
    this::createTasks);
}

From these three methods, we can derive subsequent configuration and Task registration processes. By this point, we've covered the entry content, and Library and Feature Plugins can be analyzed in a similar manner. Keeping an eye on changes in BasePlugin can provide insights into the evolutions related to AGP because many internal design modifications are mirrored in the tools or services registered in the upper-level BasePlugin. For example, many tools in AGP have gradually migrated to BuildService in recent versions, such as the GlobalScope class, which was deprecated in version 7.2. It once bundled together commonly used properties like sdkComponents.

3-4-3: Project Configuration

The configureProject() logic follows a primary thread: the registration of various public services. Note that not all services have migrated to Gradle's BuildService, and some are still ordinary Kotlin or Java classes or implementations of com.android.build.gradle.internal.services.BaseServices.

private void configureProject() {
  final Gradle gradle = project.getGradle();
  // Partial Services for Gradle's BuildService
  Provider<StringCachingBuildService> stringCachingService =
      new StringCachingBuildService.RegistrationAction(project).execute();
  Provider<...> mavenCoordinatesCacheBuildService =
      new MavenCoordinatesCacheBuildService.RegistrationAction(
                project, stringCachingService)
          .execute();
          
  ...
  
  // AGP BaseServices
  dslServices =
      new DslServicesImpl(
          projectServices,
          sdkComponentsBuildService,
          () -> versionedSdkLoaderService);
  // A plain Kotlin class          
  versionedSdkLoaderService =
      new VersionedSdkLoaderService(
          dslServices,
          project,
          () -> extension.getCompileSdkVersion(),
          () -> extension.getBuildToolsRevision());

  createAndroidJdkImageConfiguration();
}

If a service is of Gradle's BuildService type, external access can directly request cached instances from a NamedDomainObjectSet using Gradle's API:

project.gradle.sharedServices.registrations[...]

Besides, RegistrationAction is different from the commonly seen Task Action. It's merely a pattern designed by AGP to encapsulate the BuildService registration process for parameter passing and resolution.

For the BaseServices type from AGP or the ordinary Kotlin/Java type, if it's not possible to access them from the BasePlugin instance, we can consider acquiring them through reflection. For instance, if you want to access VersionedSdkLoaderService from an external Plugin, you could do the following:

// Look for a BasePlugin instance, then apply reflection
val sdkLoaderService = ReflectionKit.getField(
  BasePlugin::class.java,
  plugin, 
  "versionedSdkLoaderService"
) as VersionedSdkLoaderService

// Once acquired, you can utilize some of the helpful tools within
val btip = sdkLoaderService.versionedSdkLoader
  .get()
  .buildToolInfoProvider

// e.g. `btip.get().getPath(BuildToolInfo.PathId.AAPT2)`

The above discussion answers the second question preset in the overview. You can link this information to the "Key Points of Crafting AGP Synergy Plugins" in Section 3-2 to expand on more scenarios.

3-4-4: Extension Configuration

Our exploration starts with the configureExtension() method, a core part of AGP that can be split into two primary Tasks:

1. It sets up Domain-Specific Language (DSL) configurations such as buildOutputs, android, androidComponent, and others, which aid in crafting model classes.
2. Using the aforementioned Extension as a base, it creates associated manager classes (like VariantManager) and factory classes such as variantFactory. Here, variantFactory is an abstract interface tailored to a specific Plugin host (Application or Library). Instances of its implementation class are retained by both BasePlugin and VariantManager, enabling them to finalize the Variant's initialization and construction processes (refer to createTask).

The content about creating and using Extensions in standard Gradle Plugins can be found in Sections 2-4 and 4-2. When it comes to the Android Gradle Plugin (AGP), the process is very similar, just with a few minor differences. For information related to androidComponents, you can refer back to Section 3-3; in the context of AppPlugin, the corresponding implementation is ApplicationAndroidComponentsExtension.

The necessary Variant classes will be initialized after AGP retrieves the Extension data. Later on, when creating Tasks, we'll need to keep an eye out for whether or not they are Variant-Aware Tasks.

3-4-5: Task Creation

The createTasks() function carries two key method calls:

1. A static call to TaskManager.createTasksBeforeEvaluate().
2. An instance call to BasePlugin.createAndroidTasks().

private void createTasks() {
  configuratorService.recordBlock(
    ExecutionType.TASK_MANAGER_CREATE_TASKS,
    project.getPath(),
    null,
    () ->
  TaskManager.createTasksBeforeEvaluate(
    project,
    variantFactory.getVariantType(),
    extension.getSourceSets(),
    variantManager.getGlobalTaskCreationConfig()));

  project.afterEvaluate(
    CrashReporting.afterEvaluate(
      p -> {
    variantInputModel.getSourceSetManager()
      .runBuildableArtifactsActions();

    configuratorService.recordBlock(
      ExecutionType.BASE_PLUGIN_CREATE_ANDROID_TASKS,
      project.getPath(),
      null,
      this::createAndroidTasks);
  }));
}

The logic behind createTasksBeforeEvaluate() mainly involves registering Tasks unrelated to Android's core, such as Lint checks, device checks, test environment setup, CoreLibraryDesugaring, PreBuild, and more. As most of these Tasks aren't tied to a specific Variant, they're bulk registered in advance.

Peering into createAndroidTasks(), we can identify three primary steps:

1. Environment setup and safety checks.
2. Building Variant instances and triggering various Variant build hooks.
3. Construction of a TaskManager instance, followed by calls to createTasks() and createPostApiTasks(...) to register two Android core Task sets.

In Section 3-3, we learned about the three Variant creation process lifecycle callbacks provided by AndroidComponentsExtension: finalizeDsl(...), beforeVariants(...), onVariants(...), along with two additional APIs, registerExtension(...) and registerSourceType—that allow registration of the Variant extension data. The Callback Units registered via these five APIs are temporarily stored in VariantApiOperationsRegistrar. createAndroidTasks() and the following VariantManager#createVariantsFromCombination() method invoke the variantManager.getVariantApiOperationsRegistrar().

The connection between three lifecycle hooks and two extension data interaction APIs is illustrated below:

1. The Source Type registered by registerSourceType will be integrated into the existing SourceSets after finalizeDsl(...) and before Variant creation.
2. registerExtension unfolds in two steps: firstly, it registers DSLExtension to VariantApiOperationsRegistrar, then a configurator is triggered after creating the Variant and before the onVariants(...) callback. The newly created Variant is passed into this and accessed via variantExtensionConfig.variant.

Aside from registerExtension's basic functionality (see Section 3-3), the timing of the configurator's execution proves to be quite handy. If you ever need to operate on the Variant object before all onVariants(...), you can sneak in at this point. That answers the third question set in the previous article.

Returning to createAndroidTasks()'s final stage, regarding TaskManager#createTasks(...), you can trace the subsequent nodes to unearth AGP's core task set:

• TaskManager#createTasksForVariant(...)
• ApplicationTaskManager#doCreateTasksForVariant(...)
• AbstractAppTaskManager#createcommontasks(...)

createPostApiTasks(...) primarily establishes Tasks associated with DataBinding, Kotlin Plugins, and lifecycle Tasks such as assembleXxx or bundleXxx created as per Variant. Most AGP Tasks are paired with a CreationAction class for initial configuration, including input and output setup, InitialProvider settings as of the internal Artifacts system, and more. The design of the ComponentCreationConfig interface and its multiple subclasses offers rich information for us to trace. Feel free to delve into the Task initialization process for your preferred scenarios.

3-4-6: In Summary

By examining the above cases, I hope to empower you to fearlessly navigate AGP's complexity, transform your problem-solving approach, and embrace the "question-analysis-solution" cycle. In the next segment, we'll investigate how the Artifacts API v2 is designed and integrated into subsequent Task configurations. We'll also answer if AGP utilizes the same design strategy to manage hundreds of internal artifacts, akin to its public API.