1. What it is for
A slower checkout or a new source of errors is easy to miss when nobody opens the report after every change. The gate checks every run for you and turns the pull request red, with the reason, before the change reaches production.
jmeter -n -t plan.jmx -l results.jtl i9x-gate check results.jtl --baseline perf/baseline.json --junit gate.xml
- The same figures as the dashboard: it runs the same calculation code, so a p95 in the pipeline is the p95 you see when you open the file in the app.
- Rules per transaction and against a baseline, with a noise floor so small changes do not fail the build at random.
- It does not judge a percentile that has too few samples: it gives a warning instead of a false red.
- It writes a summary made to decide, and JUnit, JSON and Markdown reports. On GitHub Actions it also adds annotations and the job summary.
- It works offline: neither your results nor the metrics leave the CI runner.
2. Before you start
- Node.js 20 or later on the CI runner.
- A load test that writes JMeter results in CSV (
.jtlor.csv, with its header row). The recommended JMeter settings are the same as for the dashboard. - A Pro, Business or Enterprise subscription. The license key is in My account, License key.
3. Download and install
The gate is a package for npm, version 1.0.1. Download it together with SHA256SUMS, the file with its checksum. What changed in each version is in the release notes.
Check that the file is ours before installing it. The command has to say OK, or the hash has to be the same as in SHA256SUMS:
# Linux sha256sum -c SHA256SUMS # macOS shasum -a 256 -c SHA256SUMS # Windows (PowerShell): compare the hash with the line in SHA256SUMS Get-FileHash .\i9x-gate-1.0.1.tgz -Algorithm SHA256
Keep the file in your repository, for example in tools/, so every run of the pipeline installs the same version. Then install it with:
npm install -g ./i9x-gate-1.0.1.tgz
4. Your license in the CI
- Copy your key from My account, License key. Use the key of your subscription: a key tied to one computer does not work on CI runners.
- Save it as a secret of your CI called
I9X_LICENSE. On GitHub: Settings, Secrets and variables, Actions, New repository secret. You can also pointI9X_LICENSE_FILEto a file that holds it.
The gate never prints the key: the reports only show the plan and a short reference of the license. init, validate, explain and --version work without a license, so you can prepare everything first.
5. Write your rules
i9x-gate init writes an i9x-gate.json to start from. A typical configuration looks like this:
{
"version": 1,
"analysis": {"percentileMethod": "type7", "grouping": "transaction", "warmupSec": 60},
"rules": [
{"id": "global-errors", "metric": "errorPct", "op": "<", "value": 1},
{"id": "label-p95", "scope": "each-label", "metric": "elapsed.p95", "op": "<=", "value": 3000, "unit": "ms"},
{"id": "checkout-p99", "scope": {"label": "POST /api/checkout"}, "metric": "elapsed.p99", "op": "<=", "value": 2.5, "unit": "s"}
],
"baseline": {
"rules": [
{"id": "no-regression-p95", "scope": "each-label", "metric": "elapsed.p95", "maxRegressionPct": 15, "minDelta": 30, "unit": "ms"}
]
}
}- Metrics: response time (
elapsed), latency and connect time withavg,min,maxor any percentile (p95,p99.9), the error and success rates, throughput and the number of samples. Times need aunit:msors. - Scope: the whole test (default),
each-label, one label, or a pattern such asGET /api/*. Labels are the names the dashboard shows. - Prefer rules per label and against the baseline. A global p95 can pass while one transaction gets much worse, and a fixed limit set close to the usual value fails at random.
i9x-gate validatechecks the file and lists every problem at once;i9x-gate explaindescribes each metric, percentile method and exit code.
6. Save a baseline
The baseline is a summary of a run you consider good. Create it once from that run and commit it to the repository:
i9x-gate summarize results.jtl --out perf/baseline.json
- A baseline rule fails only when the new run is worse by more than
maxRegressionPctand by more thanminDelta, the noise floor. "Worse" follows the metric: more latency, less throughput. - Update it on purpose, for example after an improvement that is already in main, never automatically on every green run: otherwise a slow decline would never fail.
- It has to be computed with the same analysis settings as the rules. If they differ, the gate stops with code 2 and says what changed.
7. Add it to your pipeline
Example for GitHub Actions: it runs on every pull request and on main, installs the gate, runs the load test, checks it and keeps the reports.
name: Performance gate
on:
push:
branches: [main]
pull_request:
permissions:
contents: read
jobs:
gate:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: actions/setup-node@v7
with: {node-version: 22}
- run: npm install -g ./tools/i9x-gate-1.0.1.tgz
- run: jmeter -n -t perf/plan.jmx -l results.jtl
- run: i9x-gate check results.jtl --baseline perf/baseline.json --junit gate.xml --report-json gate.json
env:
I9X_LICENSE: ${{ secrets.I9X_LICENSE }}
- uses: actions/upload-artifact@v7
if: always()
with: {name: perf-gate, path: "gate.*"}On other CI systems the steps are the same commands. The JUnit report (--junit) is read as it is by Jenkins, GitLab (artifacts:reports:junit) and Azure DevOps.
Make it block merges
A red check marks the pull request, but GitHub still lets you merge it. To block the merge, make the gate job a required status check of main (Settings, Rules, Rulesets). On private repositories GitHub offers this only with GitHub Pro, Team or Enterprise.
8. Read the result
The summary first says how many labels fail and which ones, and then shows each change explicitly: before and after, in percent for times and in points for rates. The detail of every rule is in the JSON and JUnit reports.
FAIL no-regression-p95 [POST /api/checkout]: elapsed.p95 466.55 ms -> 738 ms (+58.2 %, worse) FAIL label-errors [POST /api/checkout]: errorPct = 4.16 % (breaks < 2 %) PASS no-regression-p95 [GET /api/catalog]: elapsed.p95 73.50 ms -> 74 ms (+0.7 %, worse)
The exit code tells the pipeline what happened. No error ends in 0: if something is wrong, the build fails.
| Code | Meaning |
|---|---|
| 0 | Pass. Warnings do not fail the build unless you ask for it. |
| 1 | A rule failed, or could not be evaluated. |
| 2 | Configuration or usage error, or a baseline computed with different analysis settings. |
| 3 | Input data error: missing or empty file, no header, too many malformed rows. |
| 4 | License error: missing, invalid or expired, a plan without the gate, or more samples than your plan allows. |
| 5 | Internal error. Run it again with --debug and send us the output. |
9. Limits and help
- Each check can read up to the rows per analysis of your plan (see limits by plan); with more, the gate stops with code 4.
- It needs about 1.7 KB of memory per sample: 2 million samples take roughly 3.3 GB on the runner.
- If something does not work, write to support@i9xperformance.com with the gate version, the exit code and the console output (it never contains your key). Response times are in the support policy.