Newman CI/CD Integration Guide

Introduction to Newman

Newman is a command-line collection runner for Postman that allows you to run and test a Postman collection directly from the command line but primarily with CI/CD setup. It’s particularly useful for:

  • Automating API testing in build processes

  • Running collections on servers without a GUI

  • Scheduling routine API checks

  • Generating test reports for documentation

Purpose

The primary goal for using Newman is to increase test coverage against services in staging and to catch issues immediately after deployment, thereby limiting bugs in production.

Required Project Structure

A typical Newman project structure includes:

/project-root
  /newman
    newman.js            # Newman configuration script
    collection.json      # Exported Postman collection
    environment.json     # Exported Postman environment
    data.json            # Test data for parameterized runs

Newman Configuration

newman.js Configuration File

A newman.js file allows you to programmatically run collections with custom settings:

const newman = require('newman');
const path = require('path');

const collection = path.resolve(__dirname, 'collection.json');
const iterationData = path.resolve(__dirname, 'data.json');
const environment = path.resolve(__dirname, 'environment.json');

newman.run({
    collection: collection,
    iterationData: iterationData,
    environment: environment,
    reporters: 'htmlextra',
    reporter: {
        htmlextra: {
            export: './reports/newman-report.html',
            omitHeaders: true,
            title: 'My Awesome Report Title'
        }
    },
    insecure: true, // Skip SSL certificate verification
    suppressExitCode: true // Won't exit due to test failures
}, function (err, summary) {
    if (err) {
        throw err;
    }
    console.log('Newman collection run complete!');
});

Important Configuration Options:

  • insecure: true - Useful for environments with self-signed certificates

  • suppressExitCode: true - Prevents Newman from failing the build even if tests fail

  • reporters: 'htmlextra' - Uses the enhanced HTML reporter for better reports

Exporting Collections and Environments

To export collections and environments from Postman:

  1. For collections:

    • Click the "…​" next to the collection

    • Select "Export"

    • Choose "Collection v2.1"

    • Select "Export"

    • Use Finder and place into project newman dir as collection.json

  2. For environments:

    • Go to the Environments tab

    • Click the "…​" next to the environment

    • Select "Export"

    • Use Finder and place into project newman dir as environment.json

Creating Data Files

Data files allow you to run the same collection multiple times with different sets of data, aka Iterations:

[
  {
    "icn": "1013981395V971305",
    "dfn983": "7227000",
    "dfn984": "552166195",
    "edipi": "2116630888"
  },
  {
    "icn": "1012845331V153043",
    "dfn983": "7216691",
    "dfn984": "552161050",
    "edipi": "1259897978"
  },
  {
    "icn": "1013120787V412913",
    "dfn983": "7219320",
    "dfn984": "552165566",
    "edipi": "1921024322"
  }
]

Use these values in your collection with {{icn}}, {{dfn983}}, etc.

Benefits of Using Data Files:

  • Test with multiple realistic data sets

  • Validate behavior across different patient records

  • Identify edge cases that might not be caught with a single test

  • Create comprehensive test coverage without duplicating requests

Setting Up Newman locally

Not necessary unless you want to run against local deployment. Otherwise, testing collection setup in staging is preferred and more efficient.

Installation

Newman requires Node.js to be installed on your system. Install Newman globally using npm:

npm install -g newman newman-reporter-htmlextra

Basic Usage

The basic command to run a collection with Newman from project root:

node newman/newman.js

This command will:

  • Execute all requests in the collection against a running service

  • Apply environment variables from environment.json

  • Iterate through test data from data.json (if configured)

  • Generate an HTML report in the reports directory

Add newman/reports/*.html to your .gitignore file to prevent reports from being committed to your repository.

CI/CD Integration

Post-Deployment Testing

For automated testing after deployment: * Jenkins shared library handles the post-deploy execution of Newman and the Slack reporter * All that is needed is a simple update to jenkins.yaml:

staging:
  runNewman: true

What Happens When Newman Runs in Jenkins:

  • Executed post-deploy if everything is configured properly in jenkins.yaml

  • Test report is posted in Slack channel #newman-test-reports

  • The pipeline, and deployment, continues regardless of test results (unless configured otherwise)

Resources

Official Documentation

Internal Resources

TODO

  • Slack channel for Postman/Newman specific questions/issues

  • Document how reports are going to be monitored and leveraged

    • Failed tests

    • 500 errors

  • SRE team or rotator roles with testing