1-3: Android Build-Tool Evolution

This section will revisit the evolution of automation builds in Android over the past decade and then answer the following questions: Why has Android selected Gradle and developed AGP as its build infrastructure? What are their present benefits?

From my viewpoint, the three fundamental issues that automation systems must address are:

• Build Configuration
• Dependency Management
• Task Orchestration

On top of that, developers desire build tools that are simpler to use, stable, and quicker to build.

1-3-1: Ant and Maven

In the early 2000s, the Apache Foundation spawned two sets of build-related tools that are well-known among veteran Java programmers:

1. Ant and Ivy
2. Maven

For the Ant and Ivy combination, Ant is responsible for build configuration management and task scheduling, while Ivy is responsible for dependency management. On the other hand, Maven is a tool that solves the aforementioned three problems together, including a repository system, a plugin system, and a predefined build process. Combined with good IDE (Eclipse, IDEA) support and a large community of plugins, it has rapidly ascended to the top of the JVM build field.

The common XML format, which has some potential drawbacks, is used as the configuration file for these tools. As project dependencies increase, the number of lines in a pom.xml file (the Maven's XML configuration) becomes widespread, and the build configuration is bloated. Adding logic and orchestrating tasks in XML is also cumbersome, as XML is a markup language, which creates a great deal of difficulty for future extensions.

Since the emergence of micro-services in the back-end field, the complexity of build configuration by individual project has decreased significantly, and the issue of XML file explosion is not obvious in this field. However, the Android build ecosystem was in disarray at the time. We observed numerous customized build flows with multiple tools and frameworks being used for development. These included the official Eclipse-based Android Development Tools (ADT), as well as Ant and Maven-based third-party scripts or plugins. At the 2013 Google I/O, the Android tools team presented the initial rationale for their search for a new build system:

• "Extensibility, Customization, and Stable APIs": An extensible and stable build toolchain that is not broken by Android SDK upgrades.
• "Unified across CI servers and IDEs": The same tools and processes can be used in IDEs and Continuous Integration (CI) servers.
• "Standard and Advanced Features": The standards that should be established to simplify the support of new features, such as custom dependency management, unit test sets, or features that developers want the Android team to customize.

I'd like to offer my own interpretation in addition:

• For all developers, it is cumbersome to use ADT on CI, which is a realistic problem, and the user experience gap results in numerous inconsistent or inconvenient situations.
• The build procedure of Maven is relatively fixed, even if the plugin API exists. It is difficult to meet the constantly changing requirements of the APK build process due to the addition of resource compilation, Dex compilation, native library integration, etc.
• Moreover, for customer-end delivery, such as package size control and code protection are substantially more complicated than the build requirements of the back-end application. An ideal platform is required to facilitate the expansion of the build process without modifying the platform itself and reduce the difficulty of the Android team and community maintenance.

1-3-2: Gradle

Against the aforementioned backdrop, the Android team selected Gradle and developed AGP and Android Studio (based on IDEA). Gradle has the following characteristics:

• Convention over configuration
• Platform-specific DSL
• More open APIs
• Efficient execution process (caching effect is generally better than Maven)

At the same time, Gradle also solved some migration or compatibility problems:

• It's compatible with Ant task execution.
• It's compatible with Maven repository management.
• It provides migration tools to assist in migrating from pom.xml to build.gradle, and the enterprise service provides the metrics of Maven build performance.

Gradle used Groovy DSL as the only supported script configuration at the beginning. The community has mixed thoughts about it: it solves the problem of XML being bulky and difficult to expand, but it also introduces new problems in that the Plugin DSL itself develops rapidly while the documentation is incomplete, and the API style looks arbitrary, resulting in a steep learning curve. Introducing Kotlin DSL (*.gradle.kts) is an effective way to ease this problem, as it offers constrained syntax, superb support for IDE auto-completion, an accessor query tool, and the same language (Kotlin) as the app's runtime feature development. For more information, please refer to Section 2-3.

AGP is built on such a solid foundation, establishing a brand-new ecosystem. It possesses greater autonomy in managing the Android build processes, deeply integrating with various Android build tools, and gradually evolving its own extension standards. Of course, the journey of AGP has faced challenges, some build tools and self-developed features integrated in the early stages of AGP have been deprecated (Figure 1.3.1). However, many of the new features in the past five years have been deeply integrated with Gradle's new functionalities (Figure 1.3.2).

AGP has been continuously striving to reduce its own learning curve and increase its openness. This can be observed through the following aspects:

• Documentation: It has migrated from the static site (GitHub Pages) to developer.android.com, with multiple historical versions and languages, as well as a better search engine.
• Sample projects: Every Android developer should have encountered a situation in which they replicated a magical Gradle code snippet from StackOverflow and it worked flawlessly, but you had no idea where it originated. The AGP team has released a new open source sample project, gradle-recipes (https://github.com/android/gradle-recipes), for the new DSL and Variant/Artifact APIs, which are beginner-friendly.
• Open ecosystem: Since 2020, the com.android.tools.build:gradle-api module has been separated, establishing its own open ecosystem and further reducing the cost of community-developed Plugins.
• Collaboration: At the end of 2020, a new AGP and AS naming rule was introduced, which synchronized the major version naming of Gradle and JetBrains IDEA. They work together to reduce toolchain version clashes for developers at the semantic level, which indirectly improves overall build stability.

1-3-3: Why is Gradle Still Used For Android Now?

Objectively speaking, there isn't a build tool available that outperforms its competitors in every way. Android chose Gradle in 2013, primarily due to the complexity of APK builds and the character of market-available tools at the time. Gradle is still in use because of the ongoing development of AGP, the affordability of numerous third-party Plugins, and the actuality of three-party cooperation (Google, JetBrains, Gradle), which is a combination of IDE, programming language, and build platform.

It is simple for many large organizations to develop a new build tool or programming language, but it requires more time and opportunity to create an entirely new community ecosystem. Similar to Dart's use of Flutter and the Spring framework for the JVM back-end field, in Android development, Gradle Plugins continue to be the most popular choice for extending configurations, Maven dependency management, or bytecode optimization tools. Maven remains active as the JVM back-end build tool; Bazel and Buck, as emerging tools, have been popular in the community (Github issues and other indicators), and their exploration is not limited to C++ or Python.

Based on the recent updates to AGP, we can foresee its rosy future through some key actions:

• AGP received a bunch of enhancements and exposures at Google I/O, Android Dev Submit, and other events in past 5 years (from 2018 to 2023).
• Document and tutorial updates included the introduction of KTS syntax, open-source samples, new architecture, etc.
• The ecosystem as a whole is maturing, most common scenarios or problems have corresponding Synergy Plugin solutions (the concept will be explained in Section 2-5), and the build performance continues to increase annually.