Improve performance of HSTSPreloadLookup::isPreloaded on linking external urls
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Umherirrender
	Aug 27 2022, 7:34 PM

Description

The SecureLinkFixer extension is looking up every non-https external linked url to be part of a list. This begins with the subdomain and if not found include also the higher level for the same domain.

To remove one level of sub domain from the url a preg_replace with non-greedy regex is used, which can be slow due to some extra work for non-greedy regex internally.

It is faster to replace the regex '/(.*?)\./' with a strpos lookup and use substr for the next substring

Details

	Subject	Repo	Branch	Lines +/-
	Improve performance of string operation on domain lookup	mediawiki/extensions/SecureLinkFixer	master	+11 -8

Customize query in gerrit

Related Objects

Mentioned In: rESLFf49dab01a196: Improve performance of string operation on domain lookup

Event Timeline

Umherirrender created this task.Aug 27 2022, 7:34 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 27 2022, 7:34 PM

Change 826918 had a related patch set uploaded (by Umherirrender; author: Umherirrender):

[mediawiki/extensions/SecureLinkFixer@master] Improve performance of string operation on domain lookup

https://gerrit.wikimedia.org/r/826918

gerritbot added a project: Patch-For-Review.Aug 27 2022, 7:35 PM

The benchmarker report for new:

Running PHP version 8.0.22 (AMD64) on Windows NT 10.0 build 19044 (Windows 10)

MediaWiki\SecureLinkFixer\HSTSPreloadLookup::isPreloaded('foobar.dev')
   count: 100
    rate: 2759410.5/s
   total:     0.04ms
    mean:     0.00ms
     max:     0.00ms
  stddev:     0.00ms
Current memory usage: 44.00 MiB
   Peak memory usage: 62.00 MiB

MediaWiki\SecureLinkFixer\HSTSPreloadLookup::isPreloaded('wikipedia.org')
   count: 100
    rate: 5115004.9/s
   total:     0.02ms
    mean:     0.00ms
     max:     0.00ms
  stddev:     0.00ms
Current memory usage: 44.00 MiB
   Peak memory usage: 62.00 MiB

MediaWiki\SecureLinkFixer\HSTSPreloadLookup::isPreloaded('not-preloaded.com')
   count: 100
    rate: 2242943.3/s
   total:     0.04ms
    mean:     0.00ms
     max:     0.00ms
  stddev:     0.00ms
Current memory usage: 44.00 MiB
   Peak memory usage: 62.00 MiB

MediaWiki\SecureLinkFixer\HSTSPreloadLookup::isPreloaded('pathological.case.that.is.not.preloaded.org')
   count: 100
    rate: 979977.6/s
   total:     0.10ms
    mean:     0.00ms
     max:     0.00ms
  stddev:     0.00ms
Current memory usage: 44.00 MiB
   Peak memory usage: 62.00 MiB

and for old:

Running PHP version 8.0.22 (AMD64) on Windows NT 10.0 build 19044 (Windows 10)

MediaWiki\SecureLinkFixer\HSTSPreloadLookup::isPreloaded('foobar.dev')
   count: 100
    rate: 2162012.4/s
   total:     0.05ms
    mean:     0.00ms
     max:     0.00ms
  stddev:     0.00ms
Current memory usage: 44.00 MiB
   Peak memory usage: 62.00 MiB

MediaWiki\SecureLinkFixer\HSTSPreloadLookup::isPreloaded('wikipedia.org')
   count: 100
    rate: 3711773.5/s
   total:     0.03ms
    mean:     0.00ms
     max:     0.00ms
  stddev:     0.00ms
Current memory usage: 44.00 MiB
   Peak memory usage: 62.00 MiB

MediaWiki\SecureLinkFixer\HSTSPreloadLookup::isPreloaded('not-preloaded.com')
   count: 100
    rate: 2139951.0/s
   total:     0.05ms
    mean:     0.00ms
     max:     0.00ms
  stddev:     0.00ms
Current memory usage: 44.00 MiB
   Peak memory usage: 62.00 MiB

MediaWiki\SecureLinkFixer\HSTSPreloadLookup::isPreloaded('pathological.case.that.is.not.preloaded.org')
   count: 100
    rate: 782519.4/s
   total:     0.13ms
    mean:     0.00ms
     max:     0.00ms
  stddev:     0.00ms
Current memory usage: 44.00 MiB
   Peak memory usage: 62.00 MiB

Change 826918 merged by jenkins-bot:

[mediawiki/extensions/SecureLinkFixer@master] Improve performance of string operation on domain lookup

https://gerrit.wikimedia.org/r/826918

Umherirrender mentioned this in rESLFf49dab01a196: Improve performance of string operation on domain lookup.Aug 28 2022, 5:20 PM

Maintenance_bot removed a project: Patch-For-Review.Aug 28 2022, 5:30 PM

ReleaseTaggerBot added a project: MW-1.39-notes (1.39.0-wmf.27; 2022-08-29).Aug 28 2022, 6:00 PM

Legoktm closed this task as Resolved.Aug 28 2022, 6:02 PM

Legoktm assigned this task to Umherirrender.