Consider moving site-wide config data out of the startup module
Closed, ResolvedPublic

Description

It just came to me today that this data isn't needed by the startup module.

History

Historically, it make sense why this is here. Originally ResourceLoader had a three-step load process: 1) startup, 2) jquery+mediawiki 3) page module queue. There our choices were fairly limited in terms of where to put this data. Choices in 2011:

  1. Bundle with the base request (jquery+mediawiki). Drawback: Frequent invalidation. Whenever the config data changes, we would effectively invalidate/redownload jQuery and the MediaWiki base library.
  2. As its own request. Drawback: Additional synchronous HTTP request. Delays user interactions due to needing to be a new synchronous step between request 1 (startup) and request 2 (jquery+mediawiki) because the base code needs these some of these configuration variables.
  3. Bundle with startup Benefit: No extra HTTP request. No extra synchronous step. And, doesn't cause (much) extra code invalidation when it changes. Only the 5 lines of "is Grade A compatible" logic, and the module tree. In 2011, deployments weren't very frequent, it was reasonable to assume a typical deployment would both change one or more config vars and the module tree. And even if not, the frequency of cache invalidation at once every few weeks was quite acceptable, given a typical browser wouldn't cache it for that long any way.

Naturally, we went with option three.

What changed?
  • In 2015 – all JavaScript became asynchronous. Including the loading of the startup module and "top queue" modules. – T107399
  • In 2018 – ResourceLoader was refactored form a three-step loader to a two-step loader. See more about this at T192623. But in a nut shell:
    • jQuery is no longer an upfront dependency. Instead, it loads as part of the page module queue, concurrently with everything else. It uses regular dependency resolution to ensure things still execute in the right order. This was made possible by making a few minor changes in mw.loader to use plain JS for some utilities instead of jQuery.
    • The "mediawiki" module was removed in favour of the bare minimum to define mw.loader being moved to startup. And the remainder (mw.html, mw.msg etc.) are now part of "mediawiki.base" which is a peer dependency just like "jquery".

We now have a renewed focus that the startup module's only job is to define mw.loader and load the modules queued on the page via RLQ (through OutputPage::addModules etc.).

Through T233675, T187207 and various other audits and perf work over the past two years, we've bunch a bunch of extraneous data from extensions out of the startup module and instead bundled with the modules that need it.

This became even easier with the introduction of Package modules (T133462)

Problem statement

We are shipping about 11 kilobytes of config data (enwiki, as of 11 Oct 2019, measured after decompression), as a synchronous part of the Startup module.

This needs to be downloaded and parsed before the Startup module can start requesting modules. None of this data is needed for mw.loader to do its job.

Proposal

Move it out of the startup module. Possibly as package file in mediawiki.base. Or perhaps as its own module. The overhead of needing a module would be paid back by saving bandwidth from startup invalidations that didn't change config data, as well as in time saved not delaying the loading of modules on all pages.

Event Timeline

Change 542711 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/core@master] resourceloader: Expose StartupModule::getConfigSettings for internal use

https://gerrit.wikimedia.org/r/542711

Change 542712 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/core@wmf/1.35.0-wmf.1] resourceloader: Expose StartupModule::getConfigSettings for internal use

https://gerrit.wikimedia.org/r/542712

Change 542712 merged by jenkins-bot:
[mediawiki/core@wmf/1.35.0-wmf.1] resourceloader: Expose StartupModule::getConfigSettings for internal use

https://gerrit.wikimedia.org/r/542712

Change 542717 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/extensions/WikimediaMaintenance@master] blameStartupRegistry: Add 'startup_conf' metric

https://gerrit.wikimedia.org/r/542717

Change 542717 merged by jenkins-bot:
[mediawiki/extensions/WikimediaMaintenance@master] blameStartupRegistry: Add 'startup_conf' metric

https://gerrit.wikimedia.org/r/542717

Krinkle renamed this task from Consider moving site-side config data out of the startup module to Consider moving site-wide config data out of the startup module.Oct 13 2019, 4:07 PM

Change 543684 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/core@wmf/1.35.0-wmf.2] resourceloader: Expose StartupModule::getConfigSettings for internal use

https://gerrit.wikimedia.org/r/543684

Change 543684 merged by jenkins-bot:
[mediawiki/core@wmf/1.35.0-wmf.2] resourceloader: Expose StartupModule::getConfigSettings for internal use

https://gerrit.wikimedia.org/r/543684

Mentioned in SAL (#wikimedia-operations) [2019-10-16T20:16:16Z] <jforrester@deploy1001> Synchronized php-1.35.0-wmf.2/includes/resourceloader/ResourceLoaderStartUpModule.php: Expose StartupModule::getConfigSettings for internal use T235350 T229836 (duration: 00m 59s)

Change 542711 merged by jenkins-bot:
[mediawiki/core@master] resourceloader: Expose StartupModule::getConfigSettings for internal use

https://gerrit.wikimedia.org/r/542711

Krinkle triaged this task as Medium priority.Oct 19 2019, 7:35 PM
Krinkle moved this task from Inbox to Accepted Enhancement on the MediaWiki-ResourceLoader board.

Change 570720 had a related patch set uploaded (by Anomie; owner: Anomie):
[mediawiki/core@master] ResourceLoader: Move site-level mw.config from startup to mediawiki.base

https://gerrit.wikimedia.org/r/570720

Change 570720 merged by jenkins-bot:
[mediawiki/core@master] resourceloader: Move site-level mw.config from startup to mediawiki.base

https://gerrit.wikimedia.org/r/570720