Skip to main content

Layered architecture for our APIs

Timeline​

TimeDescription
2023-04-17 15:36:00 WESTCreation of the document
2023-05-02 15:50:00 WESTAddition of the "Frameworks and Drivers" concept to the proposal

Status: Discussion​

Context​

Our project consists of data ingestion sinks and some ETL pipelines that produce and populate data to feed our frontend web-application and other end-user facing tools.

The connection between the frontend and the data tables is done using REST APIs. The requirements for these API include:

  • a fast read path
  • easily changeable technologies to allow migration from cloud to on-premise
  • manageable maintenance

We need to choose an architecture pattern that satisfies these requirements and provides a well-structured, maintainable, and flexible solution for our API implementation.

While our current codebase already has the foundations of proper code organization, there are still points that could be improved.

Mainly:

  • Some of our API have all the business logic living in the HTTP handlers.
  • Because of the previous point, it is not trivial to add or replace the REST layer with another network transport if the need would arise.
  • Some API objects require a lot (sometimes up to 8) arguments in their constructors, a sign that we are lacking abstraction and have too many dependencies.

Decision​

We propose adopting a hybrid architecture that combines ideas from Layered Architecture and Clean Architecture for our API implementation. This hybrid architecture will include the following layers and components:

  • Presentation Layer: This layer will be responsible for exposing the API endpoints, handling HTTP requests and responses, and validating input data. Different implementations of the same API may exist, such as both REST and gRPC interfaces.
  • Application Layer: This layer will contain the business logic. It will coordinate between the Presentation Layer and the Data Access Layer to ensure proper execution of operations.
  • Data Access Layer: This layer will handle communication with the database, abstracting the details of data storage and retrieval. It will provide an interface for the Application Layer to interact with the event data without being tied to a specific database technology.
  • Frameworks and Drivers: This layer, inspired by Clean Architecture, will consist of external libraries, frameworks, and tools that the application uses. It will help keep dependencies organized and maintainable.

Adopting this hybrid architecture offers the following advantages:

  • Separation of concerns: Each layer has a distinct responsibility, making the system easier to understand, maintain, and extend.
  • Flexibility: The modular structure allows for easy replacement or modification of individual components, such as swapping out the database technology or migrating from cloud to on-premise.
  • Maintainability: A well-organized codebase with a clear separation of concerns makes it easier to manage and maintain the system over time.
  • Testability: Individual layers can be tested in isolation, simplifying the testing process and improving overall test coverage.

Consequences​

By choosing this hybrid architecture for our API implementation, we expect the following consequences:

  1. Improved maintainability: The separation of concerns and modularity of the architecture will make it easier for developers to understand and maintain the system.
  2. Enhanced flexibility: The decoupling of components within the architecture allows for easy replacement or modification, enabling us to switch component technologies or migrate from cloud to on-premise as needed.
  3. Streamlined testing: The distinct layers and separation of concerns facilitate isolated testing, leading to a more robust and reliable system.
  4. Organized dependencies: Incorporating the Frameworks and Drivers concept will make our architecture more organized and easier to manage, as all external libraries and frameworks will be grouped together, thus reducing coupling and increasing maintainability.
  5. Increased complexity: The addition of a new layer and the combination of two architecture patterns may introduce some additional complexity to the architecture. However, the benefits of improved organization, reduced coupling, and maintainability outweigh this downside.
  6. Learning curve: Team members may need to familiarize themselves with the new architecture and its concepts, which might require some time and training.

In conclusion, adopting this hybrid architecture combining Layered Architecture and Clean Architecture concepts will provide a more organized, maintainable, and flexible solution for our API implementation while addressing the identified issues in our current codebase. This new architecture will better meet our project requirements and facilitate future changes and enhancements.