A brief introduction to the concept of CI/CD, its evolution and relevance in the context of current state of software development and the future trends around API driven development.
More...
This blog post was originally published in RapidAPI.
Do you recall how software was shipped in floppies, CDs, and DVDs in the typical brick-and-mortar fashion?
Software delivery models have undergone enormous changes in the last few decades. The Internet eased the distribution of software binaries by facilitating direct download. However, cloud computing has radically transformed the software distribution and delivery paradigm. It is now possible to run a software application without waiting for its DVD shipment to arrive or file download to complete. By leveraging the Software-as-a-Service(Saas) way, software products are accessible instantly on the web. No more binaries and no hassles of installations and upgrades.
Continuous Integration (CI) and Continuous Delivery (CD) are the main principles that define the new norms for SaaS-based software product development on the World Wide Web. This blog post covers the technical awareness of these two concepts, CI and CD, to understand how they fit into a modern software product’s hosting requirements.
What is Continuous Integration?
Real-world software development thrives on coordination. Multiple developers work on individual modules of an entire software. These modules interact with other modules or have dependencies on them. Therefore the developers of all the modules must establish a contract such that all modules integrate with each other.
Coordination between developers in software development
Continuous integration envisages a process under which developers implement incremental updates to a software product. Initially, all the code written by individual developers gets combined. The CI process ensures that this integrated code has passed all the essential checklists for business logic, coding and testing, and merged in a common repository branch.
Historically, the process of integrating various modules into a single software was done in batches of features. With the agile software development model, CI enables a seamless process. It ensures that developers work in conjunction to churn out minor improvements to the product through continual updates. Therefore the word “Continuous” assumes relevance in the CI process.
When done right, the CI resembles a process pipeline where source code representing an individual developer’s contribution is fed from one end. An integrated software product code comes out of the other end. Some of the CI pipeline’s standard tasks include code quality checks, code reviews, unit testing, and merging.
Continuous integration pipeline
As is evident from the illustration above, some CI steps are like parallel pipelines resembling the activities being performed by each developer. After all the code is merged, the pipeline becomes a single flow towards the end of the CI.
The CI pipeline’s primary goal is to foster collaboration among all stakeholders for ensuring adherence to the software development processes, quality control, and validation of all features and bugfixes being implemented. The final outcome is a software build package ready for undergoing further procedures as part of the CD.
What is Continuous Delivery?
Continuous delivery picks up from the point where the CI pipeline ends. It performs additional activities up until the final delivery of the software product.
The two most important tasks performed by any CD pipeline are:
- 1Automation Testing – While the CI pipeline ensures that the individual modules are unit tested to validate the feature requirements, it may not completely address all modules’ overall integration in the software product. Therefore, the CD pipeline performs integration testing on the CI-generated build package. It ensures adherence to all the requirements for the integrated product build. Additionally, the CD pipeline also maintains a vigil to ensure that existing product features are not impacted due to the new changes. This is achieved through regression testing.
- 2Management of Hosting Environments – As a prerequisite for hosting software, a CD pipeline must also set up and maintain different hosting environments that mimic the software product’s real configurations. These environments are spawned with different settings to assist in testing functional and non-functional requirements and deployment for specific cloud infrastructure.
Due to the stringent performance and scalability requirements for hosting a SaaS-based software product, the CD pipeline is usually split into a series of test phases to test these requirements. Each phase is associated with a hosting environment, and the CD pipeline is configured to move the software build package from one phase to the next.
The word “delivery” in Continuous Delivery implies that CD’s final outcome is to deliver the software product to a target hosting environment.
Continuous delivery pipeline
The last step in the CD pipeline is to deliver the official software release. At this point, the newly generated software build package, which previously went through the CI pipeline, is deployed on a hosting environment.
What is Continuous Deployment?
In the context of continuous software development, the acronym CD may sometimes mean “Continuous Deployment,” in addition to “Continuous Delivery.”
In a broader sense, both refer to the overall CI/CD process wherein the incremental changes in the software product are pushed into the pipeline and are incorporated into the official software release. However, continuous deployment can be interpreted differently as well.
Since SaaS hosting is the predominant way of delivering a software product, every upgrade in the product entails deploying the new software release on the hosting environment. Depending upon the software’s scale and complexity, more than one hosting environment may deploy the software. For example, an internet-scale application has to be deployed across many regional hosting environments. Similarly, customized software versions have to be deployed for different customers under their hosting environments.
Continuous deployment pipeline
All these are production deployments. However, to handle the multitude of deployment environments, the standard CD pipeline is extended to add some additional checks and balances to ensure there are no glitches at the production level. The complete pipeline comprising of the CI and CD, with the extended continuous deployment pipeline, may look somewhat like:
Combined CI/CD pipeline
Note that this flow does not capture some of the finer details of the entire process. Most significantly, some of the CI steps are repeated later in CD, like the merge and build, which are used to merge and build the source code into various repositories targeted for each official release.
What is the difference between CI and CD?
Both CI and CD work in unison to deliver an end software product that meets all the features along with quality and deployment specific needs. However, as evident from the complete pipeline depicted above, CI always comes first, followed by the CD.
If we take the analogy of a vehicle manufacturing plant, the CI is the assembly process where all cars are assembled. Subsequently, each assembled car is topped up with oils, air, and water, fitted with a battery, and taken for a final inspection and quality check. This stage is the equivalent of the CD pipeline, which involves doing a test drive of the vehicle on a specific terrain (hosting environment) to test its build and ride quality.
Although the tasks performed within both CI and CD resemble a process pipeline, there are some significant differences to keep in mind.
Parallel vs. Serial
CI is predominantly a parallel pipeline. Multiple developers are working simultaneously to deliver their code that goes through its own set of steps to be delivered to the final build package. CD is predominantly a serial pipeline wherein the software build is tested sequentially for different testing requirements. However, for the final deployment, the CD pipeline may also fan out to multiple environments.
Manual vs. Automated
The CI pipeline has many human-driven steps, such as coding and unit testing, which cannot be automated at all. Therefore most part of the CI pipeline is a manual process. In comparison, the CD is a fully automated process. It relies on repeatable, standardized steps across the testing phases for achieving the maximum quality and fastest turnaround time to a new software release.
Code quality vs. Product quality
CI is all about assimilating the granular level details of the changes in source code. These changes incorporate either a new feature or a bugfix. Therefore the focus of CI is on code quality. Whereas the CD pipeline is more focused on ensuring the overall robustness of the product functionality. In other words, the CD pipeline strives for product quality.
CI/CD in Modern Software Development
By adopting the SaaS hosted model of software delivery and agile methodologies for software development, CI/CD lends itself well to support the ever-evolving challenges of modern software development. Therefore, DevOps teams across the companies are always looking to implement more robust and efficient ways to expedite the CI/CD pipeline.
In this regard, there are a few emerging trends to watch out for:
CI/CD for Web 3.0 Apps
Web 3.0 envisages a truly democratized Internet beyond the web browser’s confines wherein humans and machines interact with each other. This approach brings in newer challenges for DevOps teams to implement CI/CD for IoT devices. This is an essential upgrade wherein the pipeline has to be extended to perform automated software updates on devices as well.
Combining CI/CD with AI To Gain Insights
Like every organizational process, the CI/CD pipeline also generates tons of data. This data is valuable in asserting the overall quality of the software and the efficiency of the process. This is another frontier that the DevOps teams can explore to bring data-driven optimizations to the CI/CD pipelines and provide deeper insights to the software development teams.
CI/CD for APIs
As we know, the world of SaaS is already shifting to an API-as-a-Product-centric approach. Under this approach, the software products are hosted as APIs. Developers then consume these APIs to build custom front-end applications. From a CI/CD point of view, this brings additional challenges because APIs have specific testing needs, and they also require continuous monitoring. By leveraging API-centric tools, the CI/CD pipelines can be upgraded to support the continuous deployment of API products.
