Skip to content

Service Descriptor

Cortex lets you describe information about your service in a YAML file that's checked into git and lives alongside the code for your service.

We believe that the metadata about your service should live with you - not locked in with us. Cortex provides useful functionality by aggregating information about all of your services and connecting with numerous third party providers, like PagerDuty for on-call details or Datadog for APM dashboards.

Because the YAML files lives along side your code, it ensures:

  • the metadata is version controlled
  • the repository where your code lives is also the source of truth for information about the service
  • you always own the data

The following sections describe how to set up the different parts of Cortex in your YAML file.

Cortex uses OpenAPI as the format for the YAML specification - we've added custom extension tags that are compatible with the official OpenAPI spec.

You can still use Cortex if you don't use OpenAPI/Swagger!

We use the OpenAPI spec as a base for service metadata since it's an open spec with official support for extensions. That lets us extend it to be a service descriptor spec with optional usage of actual OpenAPI fields.

You can use Cortex to describe REST services, kafka consumers/producers, libraries, modules within a monolith, etc. Any entity that could be described as a logical module should live in Cortex.

The advantage for you is that there's a large number of open source parsers for OpenAPI. That means if you ever decide to stop using Cortex, you can easily build on top of your existing YAML files without having to building custom parsers.

This is in line with our philosophy that you should be in control of your documentation.

Basics

The file starts by defining some high level information about the service, including the name and description.

openapi: 3.0.0
info:
  title: Braavos
  description: Data ingestion service.
  x-cortex-tag: braavos

You'll notice the x-cortex-tag property. The tag is a required property that uniquely identifies this service within your architecture. Other services will use this tag when declaring a dependency on this service.

The contact field is a standard OpenAPI field that Cortex will use for notifications. See Ownership for additional contact options.

External Documentation

Cortex is your source of truth to external documentation - instead of digging through your wikis, you can aggregate links into one easily accessible place.

info:
  x-cortex-link:
    - name: Braavos Architecture Overview
      type: DOCUMENTATION
      url: http://getcortexapp.com/wiki/engineering/bravos-arch
      description: Design doc presented to architecture committee
    - name: 5xx outage triage
      type: RUNBOOK
      url: http://github.com/cortex/wiki/runbooks/5xx

The x-cortex-link tag accepts a list of link objects. Each link has three required fields: name, type, and URL.

Field Description
name Human readable name/description of the link. This is what shows up in the UI.
type DOCUMENTATION / RUNBOOK / DASHBOARD / LOGS / OPENAPI / METRICS / HEALTHCHECK
url The actual url that the UI will link to.
description Optionally, describe what this link is.

API Documentation

API documentation is a special case. Cortex supports displaying Swagger/OpenAPI specs for each service, in the "API Explorer" tab. There are two ways of adding API specs to Cortex:

  1. Using your cortex.yaml file as an OpenAPI spec file. Since the cortex.yaml file extends the OpenAPI spec, you can simply implement the spec directly in this file.
  2. If you have existing or external specs you want to display in the API explorer (e.g. from Swagger Hub, Github raw URL, etc), simply add a URL to the spec (JSON or YAML) as an x-cortex-link, with type openapi. The spec will then show up in the API explorer.
  3. If you have specs checked into your service's Git repository, you can embed it by adding it as a relative URL with type openapi like:
    info:
      x-cortex-link:
        - name: My Spec
          type: OPENAPI
          url: docs/my-openapi-spec.yaml
    

Dependencies

One of the most powerful features of Cortex is the ability to define dependencies on other services. This powers many features, including automated notifications to service owners when a dependency deprecates its API or makes backwards incompatible changes.

info:
  x-cortex-dependency:
    - tag: braavos
      method: GET
      path: /2.0/users/{username}
      metadata:
        tags:
          - billing
          - identity
        prod: true

The x-cortex-dependency allows you to define a list of dependencies. A dependency may be directed towards a service, or more granularly, to a a specific endpoint on that service.

Field Description Optional?
tag The tag of the service this depends on. See x-cortex-tag No
method HTTP method if depending on a specific endpoint Required if path is present
path The actual endpoint (as defined in the OpenAPI file) this service depends on Yes
metadata JSON metadata tags for the relationship. Supports arbitrary objects. Yes

Service Groups

Services can be grouped (e.g. billing , identity , etc) as you see fit. This lets you apply scorecards to specific groups, instead of all your services.

info:
  x-cortex-service-groups:
    - list
    - of
    - services

Git

Cortex can pull information from your git tools, such as latest contributors, commits, and service language, as well as automatically parse cortex.yaml files on merges to master.

info:
  x-cortex-git:
    <provider>:
      repository: <user>/<repo>
      basepath: testService

<provider> is one of github | gitlab | bitbucket.

basepath (Optional, currently only supported for Github) refers to the root directory of the service. See Advanced Configuration for more details regarding Monorepos.

See Git for instructions for setting up GitHub, GitLab, or Bitbucket.

Automatic YAML parsing only works with our GitHub and Bitbucket apps, or with webhooks enabled for GitLab.

Oncall (PagerDuty / OpsGenie / VictorOps)

Cortex can fetch rotation details for your services.

PagerDuty

You can find the service ID value by going to: PagerDuty dashboard -> Configuration -> Services. The URL for the service will contain the ID, for example: https://cortexapp.pagerduty.com/services/<ID>

info:
  x-cortex-oncall:
    pagerduty:
      id: ASDF1234 # Service ID / Schedule ID / Escalation ID
      type: SERVICE # SERVICE if PD service, SCHEDULE if schedule, ESCALATION if escalation policy

OpsGenie

The OpsGenie integration supports rotations by using the Rotation UUID, or the Rotation name. The UUID for the rotation can be found in the OpsGenie dashboard.

info:
    x-cortex-oncall:
        opsgenie:
            type: SCHEDULE
            id: Cortex-Engineering

VictorOps

The VictorOps integration supports rotations by using the team ID. You'll need to connect Cortex to your VictorOps portal under the Cortex settings page, using an API key and the team slug.

info:
    x-cortex-oncall:
        victorops:
            type: SCHEDULE
            id: team-abcd12345

You can find the team ID through the Victorops portal, in the teams page, for example https://portal.victorops.com/dash/cortex-app#/team/<TEAM_ID>/users.

Embeds

Cortex can embed graphs from NewRelic, Datadog, and Grafana.

To embed a dashboard from NewRelic, copy the link to the public embed provided in the iframe snippet. Follow the instructions on NewRelic's APM documentation page to create the embeddable chart.

For Grafana charts, make sure to enable iframe embedding.

info:
  x-cortex-dashboards:
    embeds:
      - type: <datadog | newrelic | grafana>
        url: https://chart-embed.service.newrelic.com/herald/6b334ae2-b1e0-412c-b714-826a5aed8358

Ownership

Ownership is represented in the x-cortex-owners section of the YAML file. We support:

  • Okta groups
  • Slack channels
  • Email addresses

The field supports a list of owners, of varying types:

info:
  x-cortex-owners:
    # List Slack channels, *without* the preceeding "#"
    - type: slack
      channel: team-engineering
    - type: email
      email: bob@foobar.com

Emails

You can include as many emails as you'd like in the list. Each email supports an optional description field.

info:
  x-cortex-owners:
    # List of email addresses
    - type: email
      email: alice@foobar.com
    - type: email
      email: bob@foobar.com
      description: Primary point of contact

Slack channels

These channels will show up in the UI in the "Service Owners" tab, and will also be used for notifications if configured with PagerDuty.

info:
  x-cortex-owners:
    # List Slack channels, *without* the preceeding "#"
    - type: slack
      channel: team-engineering
    - type: slack
      email: team-billing

Okta

Cortex can add team information from Okta to show team names / descriptions / and team members on that team. You'll have to add an API key before this feature is enabled.

Okta API screenshot

Once you have the API key added, you can just use the name of the group, e.g. ("Everyone", "Platform", "Infra") - this is case sensitive and should be exactly what it looks like in Okta.

info:
  x-cortex-owners:
    - type: group
      name: Engineering

Google Groups

Use Google Groups as a source of truth for team memberships. You'll need to first set up the Google integration under Settings → Google.

Once the connection is set up, you can use the group email as the identifier.

info:
  x-cortex-owners:
    - type: group
      name: my-group-email@getcortexapp.com

GitHub Teams

Cortex can pull team memberships from GitHub teams. You'll have to enable our GitHub integration for this feature to be enabled.

Team name should be of the form <org>/<team> . Team names are generally converted to lowercase with - separators (Team Name would be cortexapps/team-name) — but you can verify your exact name from the permalink.

info:
    x-cortex-owners:
    - type: group
      name: cortex/engineering # Must be of form <org>/<team>

Custom Data

YAML

Cortex supports the ability to add custom data about your services in key: value format. This has a few use cases:

  • Fine grained grouping, e.g. language:java . However, we suggest using service groups for most use cases. You can search for services in the dashboard with this same format. For example, you might search language:java tier:0 billing to look for all billing services that are Tier 0, written in java.
  • Scorecards: A common use case is to have service owners fill out details about their service manually, e.g. in the case of a manual production readiness audit. You might ask something like, "Does this service have TLS 1.3 enabled?" - this would map to a key-value pair of tls-1-3: true | false .

Rules in scorecards only work for boolean or numeric metadata.

To build a scorecard on custom metadata, make sure the question is posed in a form that can be answered in boolean or numeric data. For example, tls-version: number or tls-1-3-enabled: boolean.

info:
  x-cortex-custom-metadata:
    <your key>: <your value> # number | boolean | string | list | object

Custom Data with Descriptions

You can also add an optional description field to your custom data in the YAML like:

info:
  x-cortex-custom-metadata:
    <your key>:
      value: <your value> # number | boolean | string | list | object
      description: Optional description

Descriptions will appear in the service "Custom Data" tab.

Infrastructure

Enrich your Cortex services with infrastructure information from your service platform.

Kubernetes

Cortex supports Kubernetes Deployments, StatefulSets and ArgoCD Rollouts to match cortex services to your kubernetes resources.

Cortex will automatically match resources to your service if the service tag (x-cortex-tag) and the resource name are exactly equal. The match happens across namespaces and clusters.

If the tag and resource name are not equal, you can tie a resource to a service through the namespace/name identifier and optional cluster (defined in the k8s agent's configuration).

Note: if you define this block in the YAML, Cortex will not auto-match with any resources, even if there are matches outside of the resources defined in the YAML.

info:
  x-cortex-k8s:
    deployment:
      - identifier: namespace/name
        cluster: dev
      - identifier: experiment/scratch
        cluster: dev
      - identifier: default/cortex
        cluster: prod
info:
  x-cortex-k8s:
    argorollout:
      - identifier: namespace/name
        cluster: dev
info:
  x-cortex-k8s:
    statefulset:
      - identifier: namespace/name
        cluster: dev

ECS

Cortex services can match with ECS services through their ARNs.

info:
  x-cortex-infra:
    aws:
      ecs:
        - clusterArn: abcd
          serviceArn: efgh
        - clusterArn: stuv
          serviceArn: wxyz

APM

Cortex supports fetching metrics from monitoring tools like NewRelic for scorecard rules.

New Relic

New Relic application metrics can be fetched for each service.

Instructions to find your application's id can be found in the Newrelic docs.

info:
  x-cortex-apm:
    newrelic:
      applicationId: 1234567

SignalFX

SLOs

You can create and manage SLOs by listing relevant SLIs through queries, along with a threshold and a comparator. Cortex will pull data from SignalFX, and roll up the query with a specified rollup function.

For example:

info:
  x-cortex-slos:
    signalfx:
      - query: sf_metric:"jvm.memory.max" AND area:"nonheap"
        rollup: AVERAGE
        target: 5120000
        lookback: P1Y
        operation: "<="

Field Description
query Elasticsearch query for your metric. Filter by metric with sf_metric and add additional dimensions to narrow the search. Queries resulting in multiple datasets will be rolled up according to rollup
rollup SUM / AVERAGE
target Target number for SLO.
lookback ISO-8601 duration (P[n]Y[n]M[n]DT[n]H[n]M[n]S).
operation > / < / = / <=, >=

Lightstep

SLOs

You can create and manage SLOs by listing relevant latency SLIs through Streams. Cortex will pull data from Lightstep, and track against your specified SLO. For example:

info:
  x-cortex-slos:
    lightstep:
      - streamId: sc4jmdXT
        targets:
          latency:
            - percentile: 0.5
              target: 2
              slo: 0.9995

Field Description
streamId ID of your Lightstep Stream, which can be found in Lightstep, through the URL. https://app.lightstep.com/{org}/stream/my-stream/{streamId}
percentile Percentile latency for your given streamId, out of 1
target Target latency, in ms
slo SLO percentile, out of 1

Datadog

You can add information from Datadog to display in the service home page as well as to use in scorecards.

Cortex automatically discovers SLOs and Monitors through tags.

If your SLOs and Monitors in Datadog contain a tag service:my-service-tag, where my-service-tag is the identifier in Cortex (the x-cortex-tag), the catalog will automatically connect them to your service. YAML configuration is only needed if these tags do not exist. If you use the YAML configuration, Cortex will not auto-discover from Datadog tags.

If you want to override how Cortex ties SLOs and Monitors to specific tags on Datadog, you can specify that in the YAML as such:

info:
  x-cortex-apm:
    datadog: 
      serviceTags: # List of tags & values
      - tag: service
        value: brain
      - tag: backend
        value: brain

For this YAML, Cortex will do an OR query on Datadog using service:brain OR backend:brain to find the SLOs & Monitors.

Monitors

Adding monitors let you see information about their current status directly in the service homepage. You can find your monitors at https://app.datadoghq.com/monitors/manage.

The ID of a monitor is found in the URL when you click on a monitor in your Datadog dashboard, for example https://app.datadoghq.com/monitors/**<MONITOR_ID>**.

info:
  x-cortex-apm:
    datadog: # List of monitor IDs.
      monitors:
        - 12345
        - 67890

SLOs

Datadog SLOs can be listed in the service homepage for each service. You can find the SLOs at https://app.datadoghq.com/slo.

The ID for the SLO can be found in the URL when you click on an SLO in the datadog dashboard, for example https://app.datadoghq.com/slo?slo_id=**<SLO_ID>**&timeframe=7d&tab=status_and_history.

info:
  x-cortex-slos:
    datadog: # List of SLO ids
      - id: 0b73859a3e2504bf09ad23a161702654
      - id: 228499184a9efe34d4e4e9df838c7fa1

Issue Tracking

Cortex supports fetching issue counts for your services, which you can write scorecard rules on top of.

Jira

We match Cortex services to Jira tickets with labels or components. By default, we'll match any Jira ticket that has a label matching your service's unique tag. However, you can override this by specifying labels in the service descriptor. In the case of multiple labels we'll only match issues that have all labels matching.

info:
  x-cortex-issues:
    jira:
      labels:
        - labelA
        - labelB

You can also use Jira Components to match services.

info:
  x-cortex-issues:
    jira:
      components:
        - component1

Error Tracking

Cortex supports fetching recent errors and crashes, which you can write scorecard rules on top of.

Sentry

Map your Cortex service to a Sentry project to get the list of the most recent issues in your service homepage. We map to the project's name, for example cortex-bc or my-project .

Sentry projects screenshot

info:
  x-cortex-sentry:
    project: my-project

Bugsnag

Map your Cortex service to a Bugsnag project, allowing scorecard rule creation on number of issues. We map to the project's slug. For example test or test-1 . The project slug can be found in the url. For example: https://app.bugsnag.com/$ORGANIZATION/$SLUG/errors

Bugsnag projects screenshot

info:
  x-cortex-bugsnag:
    project: test

Rollbar

Map your Cortex service to a Rollbar project, allowing scorecard rule creation on numerical RQL queries. You can also get the list of the most recent Rollbar items in your service homepage. We map to the project's name, for example FirstProject or SecondProject.

Rollbar projects screenshot

info:
  x-cortex-rollbar:
    project: FirstProject

Static Analysis

Cortex supports static analysis insights in Scorecards. This lets you build rules on data such as code smells, code coverage, and more.

Sonarqube

To begin using Sonarqube as your source for static analysis, you'll have to add a user token in the configuration page, as well as the URL to your Sonarqube host.

What if my Sonarqube instance is on our internal network?

You'll likely need to set up an on-prem installation of Cortex or whitelist our server's IP (please reach out to founders@getcortexapp.com for details)

By default, Cortex will use the service identifier (e.g. my-service) as the "best guess" for the sonarqube project name when evaluating scorecards. If you need to override this, you can use the following yaml:

info:
  x-cortex-static-analysis:
    sonarqube:
      project: my-project

Codecov

We also support Codecov. To use codecov, you'll need to use the Github tag, since Codecov maps projects by the github repository. If you have the Github repo tag set, then there's nothing else you need to do to tie a service with a Codecov project!

To set up Codecov, you'll need to get an access token and it through the settings page.

Vulnerability Tracking

Cortex supports security vulnerability insights in Scorecards. This lets you build rules on current number of detected vulnerabilities.

Snyk

To begin using Snyk, add an API token in the configuration page. We'll automatically detect all Snyk organizations visible to the API token provided.

By default, Cortex will use your service's Git repo name (e.g. myorg/myrepo) to guess matching Snyk projects across all your Snyk organizations. If you need to override this, you can use the following yaml:

info:
  x-cortex-snyk:
    projects:
      - organizationId: 01234567-e65f-4b7b-a8b1-5b642894ec37
        projectId: 01234567-e65f-4b7b-a8b1-5b642894ec37

Alerts

Cortex supports the ability to count alerts for a given time period, with custom queries.

Opsgenie

By default, Cortex will use your service's code tag to guess what alerts belong to a particular service. If you need to override this, you can use the following yaml:

info:
  x-cortex-alerts:
    - type: opsgenie
      tag: service
      value: k8s-agent-*

Where the tag refers to the property/metadata key for each alert. The value also supports wildcards to capture any alert that matches the pattern.