Back vallusby ExpandIt The version you are on is the version you are stuck with
The ISS Quest airlock module being moved by transport aircraft

Closed environments · version management

The version you are on is the version you are stuck with

Keeping Playwright current when every upgrade crosses an air gap

Playwright · upgrades · air gap

ISS Quest Airlock Module transported by NASA Super Guppy Turbine (KSC-00PP-1345).jpg - NASA, public domain (Wikimedia Commons)

Introduction: monthly releases, quarterly transfers

Playwright ships roughly every month. Getting a package into a closed environment takes somewhere between a fortnight and a quarter, depending on how the customer's transfer process works and how many approvals sit in front of it.

Those two rhythms do not fit together, and no amount of process improvement makes them fit. What you can do is decide deliberately how far behind you intend to run, and make the gap survivable. Teams that do not make that decision end up making it by accident: they stay on whatever version they installed first, until something forces an upgrade of eighteen months in one jump.


1Why upgrading is not optional

The temptation in a closed environment is to freeze. Nothing can reach the machine, the suite passes, so why touch it. Three things eventually break that position.

The browsers age with the library. Playwright pins engine builds per release. Staying on an old Playwright means testing against browsers that are further and further from what your users run, which quietly reduces what your suite is evidence of.

Security review comes back. Dependencies accumulate CVEs, and the annual review will ask about them. "We cannot upgrade" is not an answer that survives that conversation.

The jump gets more expensive. Breaking changes compound. Four minor versions is an afternoon; eighteen months is a project with its own budget.


2Choosing a cadence

Pick one deliberately and write it down. Three that work:

CadenceFitsCost
Every releaseEnvironments with a fast transfer pathConstant small effort, frequent regression runs
QuarterlyMost closed deploymentsOne planned upgrade window per quarter
On triggerVery slow transfer processesUpgrade only for a needed fix or a CVE, plus one scheduled catch-up a year

Quarterly is the default that fits most closed environments: it is frequent enough that each jump stays small, and rare enough to survive a transfer process measured in weeks.

Whatever you choose, pin exactly and commit the lock. A caret range in package.json in an environment where you cannot see what was installed is a guarantee that two machines you believe are identical are not.

json
{ "devDependencies": { "@playwright/test": "1.60.0" } }

3What actually breaks when you jump several versions

The library itself is fairly disciplined about deprecations. Most upgrade pain comes from underneath it.

Browser behaviour. New engine builds change rendering, timing and occasionally default security behaviour. Suites with implicit timing assumptions surface this as new flakiness, and it looks like your tests broke rather than like the browser moved.

Selector and API deprecations. Individually small, collectively a day of work when several releases stack up.

Report and trace format changes. If you archive traces for audit, check that the viewer you have on the isolated side still opens them. This one bites specifically in closed environments, where the viewer is also a version you shipped.

System library requirements. New engine builds occasionally need newer system packages. On a machine that cannot reach a package repository, that turns a library upgrade into an operating system conversation.

The practical consequence: upgrade in a connected environment first, on the same suite, and let it run for a week. The version that crosses the gap should be one you have already seen pass, not one you are about to find out about.


4The transfer package

Whatever the transfer mechanism is, what you hand over should be the same shape every time, including the roll
Whatever the transfer mechanism is, what you hand over should be the same shape every time, including the rollback.Imlay St Bowne St td (2018-07-07) 08 - 62 Imlay Street.jpg - Tdorante10, CC BY-SA 4.0 (Wikimedia Commons)

Whatever the customer's mechanism is, media, data diode or a supervised share, what you hand over should be the same shape every time:

playwright-1.60.0-linux-x64/
  browsers/ms-playwright-1.60.0-linux-x64.tar.gz
  packages/node_modules-1.60.0.tar.gz     # or the vendored tarballs
  deps/*.deb                              # system packages, if needed
  MANIFEST.txt                            # versions, platform, contents, date, author
  SHA256SUMS
  SHA256SUMS.asc                          # detached signature

Sign it. Not because anyone expects tampering on the way in, but because the signature is what lets the receiving side accept the package without a conversation, and that conversation is often the slowest part of the transfer.

Include the rollback in the same package: the previous version's archive, or a written note of exactly where it still lives on the isolated side. An upgrade you cannot reverse inside the maintenance window is an upgrade nobody will approve twice.


5The upgrade window on the isolated side

Sequence that survives contact with reality:

  1. Verify the package. Checksums and signature, before anything is unpacked.
  2. Install alongside, not over. Unpack the new browser cache to a path that includes the version, and switch by changing PLAYWRIGHT_BROWSERS_PATH. Both versions coexist until you delete one.
  3. Run the smoke suite. Ten minutes, covering login, one critical path and one report.
  4. Run the full regression once. Compare against the last known-good run, not against an expectation. New failures are the interesting output; unchanged failures are noise.
  5. Keep the old version for one cycle. Delete it when the next upgrade lands, not when this one succeeds.

Step 2 is what makes the whole thing reversible. Rolling back becomes an environment variable rather than a restore.


6When to stay where you are

Deliberately staying behind is a legitimate decision, and worth writing down as one:

What separates a decision from neglect is the record: which version, why, what would change it, and when it will be revisited. That record is also exactly what the security review will ask for when it notices you are four versions behind.


7Conclusion

The mismatch between a monthly release cycle and a transfer process measured in weeks cannot be resolved. It can only be managed, and managing it comes down to three things:

  1. Choose a cadence and write it down. Quarterly fits most closed environments. Not choosing means freezing by accident.
  2. Prove the version somewhere connected first. The upgrade that crosses the gap should be one you have already watched pass for a week.
  3. Make rollback a path change, not a restore. Install alongside, switch by environment variable, delete only on the next cycle.

None of it removes the lag. It turns the lag into something you chose, with a date on it, which is the difference between an answer at the audit and an apology.

Download the offline copyOne HTML file with everything inside - opens with no internet at all.