Kubernetes is widely regarded as the ideal platform for teams in large and mid-size organizations with high availability and autoscaling requirements. However, it has also added friction to existing developer workflows. This change can be understood as part of a broader push towards the adoption of the micro-service paradigm and the increasing complexity of how software is run. In this article, we will examine how the problem of software development has evolved with this paradigm shift.
Lets start with the concept of a dev loop and its role in development workflows. The development loop for any software project consists of two separate loops that intersect over git.
These are -
- Inner dev loop - The process involves writing, compiling, and testing a piece of code on a developer's own workstation. This loop continues until a satisfactory piece of code is moved to Git.
- Outer dev loop - This loop is triggered after the code has been moved from the developer's workstation to Git. It involves running tests and deploying the final product. Ideally, this process should be set up as a continuous loop, where deployments occur with every code push that passes the tests.
These loops feed into each other and form a complete software development lifecycle.
Rise of the micro-service
The advent of micro-services has brought small, independent, and loosely coupled software components that offer better scalability and reliability. We won't discuss the advantages here since its been very well publicized, but our concerns come from the consequent increased operational complexity.
Let's consider a simple setup where the
frontend calls the
api, which in turn communicates with two other services called
db. In this scenario, our developer works on the
api layer, which has both downstream (frontend) and upstream dependencies (identity, db). Let's explore how the development process for the
api service changes in the two different development loops -
Inner dev loop
- Upstream dependencies access -
apiservice needs to talk to both identity and db services to successfully run.
- Downstream dependencies access - In order to test whether the changes being made on the
apiservice are compatible with its downstream dependencies, it is also important to redirect downstream requests to this service.
Outer dev loop
- Upstream dependencies - In order for an e2e test to run for the
apiservice, there needs to be access to a running copy of
- Downstream dependencies - There is also a need for a way to allow the
frontendto access this service for testing purposes.
- Hosted endpoint - In addition to the above mentioned requirements, we also need to host this copy of the
apiservice so that it can be directly tested
The inner dev loop problem is something that
Ambassador Labs has solved for. We have attempted to solve the outer dev loop problem on the TrueFoundry platform by Intercepts.
Introducing TrueFoundry Intercepts
The concept of TrueFoundry Intercepts work by solving the three outer dev loop problems outlined in the previous sections. We follow the following strategy to tackle this -
- Preview URLs - Using the platform itself, it's easy to deploy a copy of an existing service at a specific commit SHA. This allows you to create preview URLs that are dedicated to a particular commit or pull request.
- Header based intercepts - Intercepts make it possible for a service to continue using the same URL while adding an additional header that redirects to the preview copy. For example, we can attach an intercept to the
apiservice that redirects to a preview copy based on a header being passed to it. This means
frontendcan continue to work with the same URL just by passing an additional header during testing.
We have discussed the changes in development flow that have been brought about by the advent of microservices and how we are attempting to address them. This remains an open question, and we can expect even more innovative solutions in this space in the future.
Chat with us
if you're looking to maximise the returns from your LLM projects and empower your business to leverage AI the right way, we would love to chat and exchange notes.
Learn how TrueFoundry helps you deploy LLMs in 5 mins: