Specifying Service Dependencies in Kustomize

Service Dependency db-service

Stored in https://coderepo.mobilehealth.va.gov/scm/ckm/db-service.git with the 1.0.0 tag

db-service/kubernetes/base/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: db-service-v1
spec:
  template:
    spec:
      containers:
      - name: db-service
        image: dtr.mapsandbox.net/ckm/db-service:1.0.0 # with default DTR and version tag
db-service/kubernetes/components/dev/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1alpha1
kind: Component

resources:
- ../../base # Default image is dtr.mapsandbox.net/ckm/db-service:1.0.0

configMapGenerator:
- name: db-service-app-config
  behavior: merge
  envs:
  - application.env # dev-only application config that adds to/overrides the base config

Main Service my-service

my-service/kubernetes/base/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-service-v2
spec:
  # ...
  template:
    spec:
      containers:
      - name: my-service
        image: dtr.mapsandbox.net/ckm/my-service:2.0.0 # with default DTR and version tag, but when run with skaffold, image is overridden with the skaffold-supplied value
my-service/kubernetes/components/dev/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1alpha1
kind: Component

resources:
- ../../base # Default image is dtr.mapsandbox.net/ckm/my-service:2.0.0

configMapGenerator:
- name: my-service-app-config
  behavior: merge
  envs:
  - application.env # dev-only application config that adds to/overrides the base config
my-service/kubernetes/dev-with-dependencies/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

components:
- https://coderepo.mobilehealth.va.gov/scm/ckm/db-service.git//kubernetes/components/dev?ref=1.0.0&timeout=60s
- ../components/dev

# Images:
# db-service = dtr.mapsandbox.net/ckm/db-service:1.0.0
# my-service = set by skaffold using the skaffold.yaml build config

Explanation

If the service being built is my-service, the Skaffold configuration sets the image name and tag to be built and deployed to the Kubernetes cluster, therefore any image name/tag values specified with Kustomize will be effectively ignored, but only in the context of its own project.

That allows us to set the image name and tag in the shared kubernetes/components/dev Kustomize Component to what is needed for other projects to use, without having to override it.

In the above example, the db-service sets the "default" image in its base overlay to dtr.mapsandbox.net/ckm/db-service:1.0.0. This value propagates to the dev Component, which is referenced in my-service’s `dev-with-dependencies overlay.

As with my-service, when building db-service with skaffold, the base image value is ignored and is set only as configured in its skaffold.yaml file.

Kustomize Localize

localize is an alpha feature that fetches Kustomize remote git service dependency references and stores the referenced repo locally in order to avoid repetitive expensive network operations for every kustomize build execution. This helps speed up iterative and CI-based builds and is a feature our framework relies on.

This command is called in the scripts/localize.sh script before the Skaffold "Render Manifest" phase as a manifest/hooks/before hook. The output of this script is stored in the localized directory. The script iteratively calls kustomize localize for each kustomize overlay that is provided and is only necessary for overlays that reference remote dependency git configurations.

For each directory/overlay specified, a SHA-256 hash is generated based on the content and stored in .kubernetes-hash.sha256. It’s then checked before subsequent localize executions and only re-fetched if any of the source content changes.

To force regeneration of the localized directory, run the Skaffold localize profile (skaffold dev -p localize) or run localize.sh directly (./scripts/localize.sh force).

You should not normally delete or modify files in the localized directory because this will cause this directory to not correctly correspond to the "source" Kustomize configurations and will not be re-synced unless it is forced. It may even break the deployment. If you do edit these files (for instance, to quickly troubleshoot or debug your deployments), make sure to force re-localization after you’re done, via one of the commands described above.