Maniphest T204881

Failed executing job: AssembleUploadChunks "Disallowed by AbuseFilter"
Closed, ResolvedPublicPRODUCTION ERROR
Actions

Description

Error

Request ID: W6KknQrAEHIAAHs3YOAAAADC

message

Failed executing job: AssembleUploadChunks File:###.##.webm filekey=###.##.webm filename=###.webm requestId=W6I1yQrAAC4AAAPF@1QAAAAD session={"headers":"array(...)","ip":"#.#.#.#","sessionId":"###","userId":#}

Sample 1

{| style="width:100%; padding:1px 4px; border:1px solid #bb7070; background:#ffdbdb;"
| style="text-align:center; width:60px;" |[[File:Dialog-error.svg|50px|link=|Your action has triggered the Abuse Filter]]
| An automated filter has identified this edit as potentially unconstructive, and it has been disallowed. If this edit is constructive, please [[Commons:Abuse filter#False positives|report this error]].
|}

Sample 2

'''This upload has been automatically identified as harmful and has been disabled.'''

Notes

Logstash has recorded this job as failing 112 times in the last 30 days, since at least 1.32.0-wmf.16, all of which cary the wiki: commonswiki tag.

Details

	Subject	Repo	Branch	Lines +/-
	concatenateChunks failures are not job failures	mediawiki/core	master	+4 -2

Customize query in gerrit

Related Objects

Mentioned In: T240281: Clarify confusion between Multimedia and Structured Data team and update docs accordingly
T212082: Do not block page moves with AbuseFilter on LocalRenameJob page moves
Mentioned Here: T212082: Do not block page moves with AbuseFilter on LocalRenameJob page moves

Event Timeline

Krinkle created this task.Sep 19 2018, 8:27 PM

Restricted Application added a project: Multimedia. · View Herald TranscriptSep 19 2018, 8:27 PM

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Krinkle updated the task description. (Show Details)Sep 19 2018, 8:29 PM

This seems to be due to a flaky AbuseFilter on commons. However, I don't know what this job is supposed to do and why such filter is triggered.

https://commons.wikimedia.org/w/index.php?title=MediaWiki:Abusefilter-disallowed&diff=prev&oldid=321252572

Daimona mentioned this in T212082: Do not block page moves with AbuseFilter on LocalRenameJob page moves.Dec 16 2018, 1:36 PM

Happened 11 times in the last 30 days, and now we have the filter ID. Given that it is public, I'm gonna say that this happen due to filter 192. Shares the cause - and possibly the solution - with T212082.

• mmodell changed the subtype of this task from "Task" to "Production Error".Aug 28 2019, 11:08 PM

Krinkle mentioned this in T240281: Clarify confusion between Multimedia and Structured Data team and update docs accordingly.Jun 23 2020, 7:02 PM

Recent occurrences are indeed all caused by filter 192, according to the filter's purpose.
So... given that these failures are actually valid and intentional, do we want to keep them, or treat AbuseFilter rejections as "not an error" (even though the actual goal was not achieved)?

@matthiasmullie AbuseFilters should not apply during job invocations, however bypassing must be done with care to ensure the filters are accomodated earlier in the process. Otherwise it would indeed allow undesired actions to be taken.

When a job fails, it is considered a system failure in our software. For some jobs a retry is allowed. But after that it is generally irrecovable. There is no feedback back to the user as far as I know. When an abuse filter error happens during the Edit or Upload API actions from UploadWizard, or on a regular wiki action page, this feedback is handled by the GUI in question and turned into something useful that the user can understand and act on accordingly. For example, by choosing a different title, or differnent license etc whatever it might be that the filter flagged.

The filtereable criteria for a user upload are hopefully applied during intake. Then the job shouldd run on that same authoritity to save regardless.

Perhaps a next step here could be to understand why these same filters did not trigger earlier in the GUI. Or if they did, why it was still queued as a job? Or perhaps some of the information is missing in the initial run? Or perhaps there is someting generic/faulty in the job context that wrongly causes the job to be caught by a filter?

These jobs are only triggered for async chucked API-based uploads.
The abuse filters can't be executed earlier in the process - running them on individual chunks makes no sense, and this job is where they are recombined into the actual file.
The result status does feed back to the API, though (stored in session - next API poll request will be informed about success/failure status)
So yeah, the more I think about it, the more I'm convinced that the entire existing process is just fine, except that the job shouldn't be considered a (software) failure in this case; it was a rejection, and those details are already being passed along to the API clients - i.e. the job has completed successfully.

Change 608886 had a related patch set uploaded (by Matthias Mullie; owner: Matthias Mullie):
[mediawiki/core@master] concatenateChunks failures are not job failures

https://gerrit.wikimedia.org/r/c/mediawiki/core/ /608886

gerritbot added a project: Patch-For-Review.Jul 1 2020, 2:28 PM

matthiasmullie added a project: Structured-Data-Backlog (Current Work).Jul 1 2020, 2:29 PM

matthiasmullie moved this task from Incoming to Code Review on the Structured-Data-Backlog (Current Work) board.

Cparle moved this task from Code Review to Verify on Production on the Structured-Data-Backlog (Current Work) board.Aug 10 2020, 12:08 PM

Change 608886 merged by jenkins-bot:
[mediawiki/core@master] concatenateChunks failures are not job failures

https://gerrit.wikimedia.org/r/608886

ReleaseTaggerBot added a project: MW-1.36-notes (1.36.0-wmf.4; 2020-08-11).Aug 10 2020, 1:00 PM

Maintenance_bot removed a project: Patch-For-Review.Aug 10 2020, 1:11 PM