Skip to content

Executor Azure DevOps⚓︎

Overview⚓︎

The Executor Azure DevOps is an Executor Adapter that acts as a proxy between the DevOps Server and Azure DevOps.

Its purpose is to receive all the Tasks to execute, process them, and forward the execution request to Azure DevOps. In addition to this:

  • it keeps track of the mapping between the ODM Task and the Azure DevOps pipeline that has been triggered,
  • it exposes an endpoint to check the pipeline status,
  • it updates the mapping properties with status information.

In order to properly interact with Azure DevOps, it needs to handle the authorization mechanism; for this reason, the Executor Azure DevOps microservice uses OAuth 2.0 to get the authorization credentials needed for Azure services.

How it works⚓︎

Architecture⚓︎

As the majority of the ODM services, the Executor AzureDevOps Service is composed by:

  • Azure DevOps API module: a module containing resource definition and a client to interact with Azure DevOps
    • differently from the Product Plane services, it does not expose any abstract controller
  • Azure DevOps Server module: a module implementing the abstract controller, any other component to interact with the DB, and any service needed for the logic
    • the abstract controller and commons part are not retrieved from the API module, but from the Executor module of the Utility Plane

The Executor AzureDevOps is an adapter, an implementation of Executor, so it only overrides parts specific to Azure DevOps, including communication with it and handling resources.

Executor-AzureDevOps-diagram

Relations⚓︎

Executor AzureDevOps it's not meant to be a standalone service, but it has the ability to be used as an independent module. It exposes reachable APIs, and it includes any information needed to communicate with Azure.

Even if it's possible, the only actions that don't involve any other ODM services are access to the adapter Database to retrieve information about Task executions.

Its default usage always includes interactions with: * DevOps Service, to receive Task to be executed and to, eventually, check the status of the pipeline once the Task returns a callback to the DevOps server * Azure DevOps, to request pipeline executions or information about a specific run of a pipeline

Tools⚓︎

OAuth 2.0⚓︎

OAuth 2.0 is the authorization protocol used for interactions with Azure DevOps. It adds all the necessary authorization information to any request forwarded to the DevOps provider through an Access Token. It uses a client credential flow to interact with the Microsoft Entra identity provider to get the Access Token.

Upon start-up, the Executor DevOps must be configured through the property files with OAuth parameters like:

  • Token URI (i.e., the identity provider URI to get the authorization token)
  • Client ID
  • Client Secret
  • Scope

Examples⚓︎

Azure DevOps Task configuration⚓︎

Every Executor Azure DevOps task involves an interaction between an ODM Task (configuration of a single step in the data product lifecycle) and an Azure DevOps pipeline (execution of the configured step on the real infrastructure). Any information required to identify the right pipeline is encoded in the input Task itself, as detailed in the DevOps Microservice documentation page, specifically in the lifecycle attribute of the internalComponents attribute of the Data Product descriptor.

After the Task creation, the adapter for AzureDevOps requires the Task object to have a template attribute, containing:

  • organization
  • project (i.e., project ID)
  • pipelineId
  • branch

and a configurations attribute, containing:

  • stagesToSkip: a List with the name of stages of the desired pipeline to skip
  • params: a Map, a set of key-value pairs representing the parameters required from the pipeline in order to be executed

Technologies⚓︎

In addition to the technologies already mentioned on the main architecture page, this module makes use of the following:

References⚓︎