Filegen 6.0 Migration Guide

This guide covers the migration from ngss-filegen-maven-plugin 5.x to filegen 6.0, which introduces Copier-based template generation for DevOps and Backstage configurations.

Overview

Filegen 6.0 introduces Copier-based generation for DevOps and Backstage templates, while legacy goals continue to work for other file types.

What Uses Copier (New)

Template Init Goal Update Goal

DevOps (Kubernetes, Skaffold)

filegen:copier-init -Dcopier.template=devops

filegen:copier-update -Dcopier.template=devops

Backstage (catalog-info.yaml)

filegen:copier-init -Dcopier.template=backstage

filegen:copier-update -Dcopier.template=backstage

What Uses Legacy Goals (Unchanged)

File Type Goal

Version Validator (VV)

filegen:generate-vv

DIBR configs

filegen:generate-dibr

Why Copier? Static vs. Dynamic Templates

Filegen 5.x used static template generation: files were generated once and never updated. This created significant maintenance challenges over time. Filegen 6.0 adopts Copier-based dynamic templates which fundamentally change how configuration files are managed.

Aspect Static Generation (5.x) Copier Templates (6.0)

Updates

Overwrites all files, then manually revert unwanted changes

Automatic - copier-update merges changes intelligently

Customizations

Lost on regeneration

Preserved via 3-way merge

Bug fixes

Require manual intervention across all services

Roll out via template update

Consistency

Drift over time as teams make ad-hoc changes

Convergence via shared template

Audit trail

None - no record of original template

.copier-answers/ tracks template origin and version

Best practices

Frozen at generation time

Continuously improved

The Real-World Problem

Consider a typical scenario with static generation:

  1. You generate Kubernetes configs for 20 services in January

  2. In March, you discover a security issue requiring a label change

  3. In May, the team adds a new Skaffold profile everyone needs

  4. In July, resource limits need adjustment based on production data

With static generation, each change requires:

  • Re-running filegen, which overwrites all generated files

  • Manually reverting unwanted changes file-by-file using git diff/git checkout

  • Risk of accidentally losing intentional customizations

  • Each update becomes increasingly tedious as customizations accumulate

With Copier templates:

# Fix is added to template once, then for each service:
mvn filegen:copier-update -Dcopier.template=devops
# Review changes, resolve conflicts, done

Key Benefits

Updateable Configurations

Re-run updates to incorporate template improvements without regenerating from scratch. The template repository becomes a living source of best practices that all services can adopt.

Intelligent 3-Way Merge

Copier compares three versions: (1) the original template when you first ran it, (2) your current files with local modifications, and (3) the new template version. This allows it to:

  • Automatically merge non-conflicting changes

  • Preserve your intentional customizations

  • Only prompt you when there’s a genuine conflict

    This is the same algorithm Git uses for merges, applied to template management.

Auto-Detection from pom.xml

The template reads your pom.xml to automatically populate:

  • artifactId → service name, image name, deployment name

  • groupId → namespace hints

  • version → image tags

  • description → Backstage catalog descriptions

    This eliminates manual entry errors and ensures consistency.

Version Pinning and Rollback

The .copier-answers/devops.yaml file records exactly which template version was used:

_commit: abc123def
_src_path: https://coderepo.../ngss-devops-template.git

This enables:

  • Reproducing the exact configuration at any point

  • Updating to a specific version: copier-update -Dcopier.version=v2.0.0

  • Rolling back if a new version causes issues

Team Collaboration

When everyone uses the same template:

  • New team members see familiar patterns across services

  • Code reviews focus on business logic, not boilerplate differences

  • Onboarding is faster ("just run copier-update")

  • Template improvements benefit everyone

Separation of Concerns

Infrastructure best practices live in the template repository, not scattered across service repos. The DevOps team can improve the template, and service teams adopt changes at their own pace.

Prerequisites

Before migrating, ensure you have:

  • Copier 9.17.0+ installed (pip, Docker, or Maven)

  • Git repository with all changes committed (copier requires clean working directory)

Migration Steps

1. Install Copier

# Recommended: pipx
pipx install copier

# Or pip
pip install copier

# Verify
copier --version  # Should be 9.17.0+

See Copier Installation Guide for Docker and other options.

2. Commit Current State

Copier requires a clean git working directory:

cd my-service
git add -A && git commit -m "Pre-migration state"

3. Run Copier Template

Run the DevOps template on your existing project:

copier copy https://coderepo.mobilehealth.va.gov/scm/ckm/ngss-devops-template.git . --trust

The template will:

  • Auto-detect settings from your pom.xml (artifactId, groupId, version)

  • Detect existing configurations where possible (Dynatrace labels, etc.)

  • Prompt for new required values (VAEC labels, profile selections)

4. Answer VAEC Label Questions

The template prompts for VAEC cost allocation labels (required for AWS EKS):

🎤 VAEC VASI (VA System Identifier): <your-vasi>
🎤 VAEC Product Line: <your-product-line>
🎤 VAEC Service Shortname: <your-service-name>
🎤 VAEC CKM ID: <your-ckm-id>
🎤 VAEC Product: <your-product>
🎤 VAEC Application Name: <your-app-name>
If you have existing Dynatrace labels, these values may be auto-detected.

5. Select Skaffold Profiles

Enable the profiles your project needs:

🎤 Include 'dev-with-dependencies' profile? [Y/n]:
🎤 Include 'vault-test' profile? [Y/n]:
🎤 Include 'debug-all' profile? [Y/n]:
🎤 Include 'staging-build-test' profile? [Y/n]:
🎤 Include 'map-staging-deploy' profile? [Y/n]:
🎤 Include 'map-sandbox-deploy' profile? [Y/n]:
🎤 Include 'map-prod-deploy' profile? [Y/n]:

6. Review and Resolve Conflicts

Copier uses 3-way merge to intelligently handle conflicts between your existing files, the old template version, and the new template version.

For each conflicting file, Copier will prompt you:

Conflict in kubernetes/components/dev/overrides.env:
  [y] Overwrite with new template version
  [n] Keep your current version
  [d] Show diff between versions
  [e] Edit file manually to resolve

Recommended approach:

  1. Press d to view the diff first

  2. If your customizations should be preserved, press n or e

  3. If the template change is needed, press y

Common files with customizations to watch for:

  • kubernetes/components/dev/overrides.env - Environment variables

  • kubernetes/dev-with-dependencies/kustomization.yaml - Service dependencies

  • skaffold.yaml - Custom build/deploy configurations

7. Validate and Test

# Validate skaffold configuration
./scripts/validate-skaffold.sh

# Test local development
skaffold dev -p dev

# Or with dependencies
skaffold dev -p dev-with-dependencies

8. Commit Migration

git add -A
git commit -m "Migrate to filegen 6.0 / Copier templates

- Added Copier-based DevOps generation
- Added VAEC cost allocation labels
- Added .copier-answers/devops.yaml for future updates"

Directory Structure Changes

Old Structure (5.x)

kubernetes/
├── overlays/
│   ├── local/
│   ├── staging/
│   └── prod/
└── base/

New Structure (6.0)

kubernetes/
├── base/                      # Base manifests
├── components/                # Reusable components
│   ├── dev/
│   ├── secrets/
│   ├── vault/
│   └── ...
├── dev-with-dependencies/     # Profile overlays
├── sqa/
├── sandbox/
├── prod/
└── ...

Maven Plugin Changes

DevOps Goals

5.x 6.0

mvn filegen:generate-skaffold

mvn filegen:copier-init -Dcopier.template=devops

mvn filegen:generate-kubernetes

mvn filegen:copier-init -Dcopier.template=devops

(manual regeneration)

mvn filegen:copier-update -Dcopier.template=devops

Legacy Goals (Still Available)

These goals remain unchanged in 6.0:

mvn filegen:generate-jenkinsfile  # Jenkinsfile generation
mvn filegen:generate-vv           # Version Validator
mvn filegen:generate-dibr         # DIBR configs

Updating After Migration

Once migrated, use Copier’s update command to incorporate template improvements:

# Check for updates
copier check-update -a .copier-answers/devops.yaml .

# Apply updates (with 3-way merge)
copier update -a .copier-answers/devops.yaml

# Re-answer specific questions
copier update --ask version --ask vaec_vasi -a .copier-answers/devops.yaml

# Update all Copier templates at once
mvn filegen:copier-update

Copier’s 3-way merge will:

  1. Compare your current files with the original template version you started from

  2. Compare with the new template version

  3. Automatically merge non-conflicting changes

  4. Prompt you to resolve actual conflicts

VAEC Labels Reference

The following labels are added to all deployments for AWS cost allocation:

Label Purpose

vaec-map/ckid

CKM project identifier

vaec-map/vasi

VA System Identifier

vaec-map/product

Product name

vaec-map/product-line

Product line grouping

vaec-map/application-service

Service shortname

vaec-map/application-name

Full application name

These labels appear in both metadata.labels and spec.template.metadata.labels of deployments.

Troubleshooting

"Destination repository is dirty"

Copier requires a clean git working directory:

git status
git add -A && git commit -m "WIP"
# or
git stash

"Template requires trust" Error

Configure trusted repositories or add --trust:

copier copy <template> . --trust

See Trusting Templates for permanent configuration.

Skaffold Profile Not Found

Ensure you enabled the profile during template generation. To add it later:

copier update --ask include_profile_dev_with_dependencies -a .copier-answers/devops.yaml

Recovering from Bad Merge

If you made a mistake during conflict resolution:

# Reset to pre-migration state
git checkout HEAD -- .

# Re-run copier
copier copy https://coderepo.mobilehealth.va.gov/scm/ckm/ngss-devops-template.git . --trust

Missing Auto-Detected Values

If values weren’t auto-detected, provide them manually:

copier update --ask artifact_id --ask group_id -a .copier-answers/devops.yaml