In T369788#9977651, @DMartin-WMF wrote:

I agree. It seems to me it should always just be the orchestrator time.

I agree too. Even if we capture pre-orchestration and post-orchestration latency in a true end-to-end elapsed time, I think we would still be more interested in the period from the start of orchestration to its end.

In T363908#9964007, @DMartin-WMF wrote:

…will comment on them tomorrow.

In T363908#9952591, @DMartin-WMF wrote:

If the team agrees with it, am I right in thinking that you would also approve of this idea of changing the strategy to rely on runtimes?

@DMartin-WMF I note it says in the document (towards the end) “…the Z8K4 property is only updated if the average runtime for the new best performer is significantly better than the average runtime for the previous best performer (and the new best performer has no more test failures than the previous best performer).” Can you perhaps clarify whether “…no more test failures…” means “…no more failures of connected tests…”? Thank you!

https://www.wikifunctions.org/view/en/Z13533 (Python) jumped to the top of the list and got disconnected.

In T363908#9947799, @DMartin-WMF wrote:

… I wrote a doc page outlining how the implementation list ordering happens.

Random evaluations:
22:00 UTC
✅https://www.wikifunctions.org/view/en/Z15386
✅https://www.wikifunctions.org/view/en/Z17105
❌https://www.wikifunctions.org/view/en/Z14226 Service Unavailable [passing at 22:50]
✅https://www.wikifunctions.org/view/en/Z10869
❌https://www.wikifunctions.org/view/en/Z11208 Service Unavailable [persistent]
✅https://www.wikifunctions.org/view/en/Z11684
22:30 UTC
✅https://www.wikifunctions.org/view/en/Z805 (equality function changed in one test)
✅https://www.wikifunctions.org/view/en/Z14304
23:00 UTC
❌https://www.wikifunctions.org/view/en/Z13633 Service Unavailable [passing at 00:30 UTC]
✅https://www.wikifunctions.org/wiki/Z808
⚠️https://www.wikifunctions.org/view/en/Z15251 Three Python implementations but only one passes (Z15377), otherwise Service Unavailable [Z15621 passing after 00:00 UTC; Z15252 has some passes too and is disconnected, probably because it fails for large values – errors are meaningful]
✅https://www.wikifunctions.org/view/en/Z17111 (with one odd exception: Z17117 with Z17114)

Please see https://www.wikifunctions.org/view/en/Z16688

Sounds good, but I wonder why this is Medium priority when T363908 is Low priority.

In T367294#9891741, @gengh wrote:

I've observed this behavior and located its origin. I believe this will be fixed when we merge https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikiLambda/+/1036184

Maybe we could take another look at this?

In T360169#9879437, @AAlhazwani-WMF wrote:

In T360169#9879412, @GrounderUK wrote:

Ah, okay… thanks. It looks like T367139 was merged/closed in error?

thank you, i just wanted to be extra sure :) reading through T367139 description, i do think that task is a duplicate of this one.

Ah, okay… thanks. It looks like T367139 was merged/closed in error?

I [do not] object to this change [see below]

Effect of “Recent changes”:

Also: https://www.wikifunctions.org/view/en/Z16559 (a standalone function call evaluates as expected)

In the Z801 data, the page IDs are unquoted; in the Wikifunctions:Status data, they are quoted 🤷‍♂️

🤔Why isn’t the page ID quoted?

*https://www.wikifunctions.org/view/en/Z16573 is a test case linked directly to Z828. The failure is similar but the Z99K1 is not an empty object.

In T366246#9845474, @Aklapper wrote:

@GrounderUK: Assuming this task is about Wikifunctions, hence adding that project tag so other people can also find this task when searching via projects or looking at workboards. Please set appropriate project tags when possible. Thanks!

Thank you. I thought I had added the team but apparently not.

I think this is T359233 again. The month objects in a list are not converted to integers, so, naturally enough, they are not equal to an integer. I don’t think Z16536 passes at all; it just says it passes while its metadata (currently) shows a timeout for both implementations (which is a separate defect). I think the recursive composition ultimately depends on a Python implementation, so I doubt if has ever passed (for Gregorian calendar months).

Now also applies to Gregorian calendar month. Please see https://www.wikifunctions.org/view/en/Z16538 and https://www.wikifunctions.org/view/en/Z16555. The first passes by special-casing (“almighty hack”) and the second passes as a result of an amended expectation (it should be False but is set to True).

Also affects Gregorian calendar month, of course. Please see https://www.wikifunctions.org/view/en/Z16555

In T364098#9771069, @Tohaomg wrote:

JS is unable to run an implementation of "English cardinal" function (while it works perfectly in a browser console): https://www.wikifunctions.org/view/en/Z15955

(…and Python is fine)

Sorry, could be P1!

Please also see the function “not”https://www.wikifunctions.org/wiki/Z10216?action=history&uselang=en

May I chip in with a question?

Consider making this a Z2 key so that any persistent object can reference a corresponding Wikidata item.

I’ve created a new function to facilitate testing of Natural number results:
https://www.wikifunctions.org/view/en/Z15914

Please see https://www.wikifunctions.org/view/en/Z15907
A simple test with equality of Natural numbers (https://www.wikifunctions.org/view/en/Z13522) times out with the correct actual result. A second test was failing but now passes after conversion to object equality (https://www.wikifunctions.org/view/en/Z13052)

Probably a separate request but I thought I’d mention it here first: it would also be useful if we could save an existing test case as a new one, while editing it (leaving the original unchanged). Not quite as useful, but perhaps more intuitive, would be the ability to duplicate an existing test case and amend it before publishing it.

In T362278#9747439, @GrounderUK wrote:

I started to raise a Bug Report for this a while back but never got round to filing it… sorry! I’ll get to it eventually (there are other issues, which is why I never finished it).

✅Done: T363623

In T363623#9751476, @99of9 wrote:

I made this as a Test Case. https://www.wikifunctions.org/view/en/Z15886

Good idea, thanks. I made:
*https://www.wikifunctions.org/view/en/Z15903 for a Boolean (passes)
*https://www.wikifunctions.org/view/en/Z15904 for a Natural number (fails: Z507 in result validation using Z13052/object equality). I was assuming that a reference was being returned and then expanded but I have a hunch that Natural number types are expanded before Z803 is executed and Z803 is simply returning the type object that it finds associated with the Z1K1. Given the apparent performance overhead with expanded types (if my hunch is correct), I think it might be better to continue with the “inconsistency”. Attempting to extract the Z4K1 from the Z1K1 of a Natural number currently times out, however (Z574), so I have left the test case failing (and consistent).
*https://www.wikifunctions.org/view/en/Z15905 for a test case (fails: Z507 with Z511 in Z507K2 in actual result)

In T358589#9704637, @cmassaro wrote:

Hello! For the moment, it looks like we will leave this task alone. The issue here is the backend validation workflow, which we've intentionally deactivated in certain circumstances. Re-enabling it isn't a high priority right now.

I don't have any answers about frontend type detection; I will leave that for @DVrandecic to think about.

In T362278#9720497, @cmassaro wrote:

type of should be possible with Z803/Value by key. I am not claiming this would be good or efficient, but, in theory, you should be able to get to

equal type(
    type of( K1 ),
    Boolean
)

with

equal type(
    value by key(
        Z39[ Z39K1: Z1K1 ],
        Z18[ Z18K1: K1 ]
    ),
Boolean )

Thanks for the tip, Cory… I think there’s a bug in Z803. I would expect that it would succeed for any object, since every object has a Z1K1, but it fails for some types, notably strings. (Z500K1
"Call tuples failed in returnOnFirstError. Error: TypeError: responseEnvelope.Z22K1.asJSON is not a function." – which looks like an error handling an error…)

My guess here is that this is not an error at all; there is simply a timeout in the comparison of the (correct) actual results with the (correct) expected results. We could clarify this by having a different error according to where the timeout occurs, but if it occurs before the comparison, we get an error in the actual result, so then they’re not the same.

In T359664#9709817, @AAlhazwani-WMF wrote:

In T359664#9709254, @GrounderUK wrote:

Thank you 🙏

In T359664#9706260, @AAlhazwani-WMF wrote:

…we also explored a drag-and-drop interaction…

That looks pretty classy!
🤔…but a purely drag-and-drop solution has accessibility implications, so “insert” should be the priority.

totally agree! that was one of the reasons that made us pause a little longer on drag-and-drop, and wait for a more solid and accessible solution from Codex. as far as i can tell there are ways to make this interaction accessible via keyboard-only[1][2], but we'd like to test this extensively, and leverage the design system team extensive know-how on a11y best practices.

[1] https://dev.opera.com/articles/accessible-drag-and-drop/
[2] https://pressbooks.library.torontomu.ca/wafd/chapter/sortable-lists/

Thank you 🙏

In T359664#9706260, @AAlhazwani-WMF wrote:

…we also explored a drag-and-drop interaction…

That looks pretty classy!
🤔…but a purely drag-and-drop solution has accessibility implications, so “insert” should be the priority.

In T362278#9708152, @DVrandecic wrote:

…The implementation in https://www.wikifunctions.org/view/en/Z15723 should obviously not exist: it is merely a wrapper around the function itself

Please consider supporting “insert item”. It looks like adding a new item near the start of a list would not be a great experience otherwise.

In T361654#9695734, @cmassaro wrote:

Hmm, okay. So what are the ZIDs of a function and implementation that should currently work but don't? How can I reproduce the bug right now?

So I created a sandbox composition with echo, and this works correctly. https://www.wikifunctions.org/view/en/Z15160

I created a comparable error with a sandbox implementation here: https://www.wikifunctions.org/view/en/Z15106.

Yes, it is strange. When https://www.wikifunctions.org/wiki/Z14717?uselang=en&oldid=95554 (which is mapped as you would expect) produced erroneous results, I changed the composition so that all eight elements were mapped from the first argument (set to True). Then I set the second argument to False and tried changing the mapping for each element in turn. This confirmed that when its value was mapped into any of the first three elements, the implementation evaluated correctly to False, but if its value was mapped to any element from the fourth to the eighth, the implementation evaluated incorrectly to True.

In T358589#9690869, @cmassaro wrote:

…For user-defined types and generic types, this will be true unless and until we decide to banish arbitrary function calls from the type calculus.

In T358589#9690869, @cmassaro wrote:

…I'll wait for others weigh in.

In T361654#9690806, @cmassaro wrote:

Ah! I see. I was running it with all of the Boolean argument values set.

In T290595#9690130, @Amire80 wrote:

Which problem does this solve? I can more or less guess some, but it needs a clearer definition.

In T360904#9689310, @Jdforrester-WMF wrote:

Now deployed, so should be able to re-test and get further details.

This function has seven implementations and five tests. All seven implementations pass all of the tests and each test passes all seven tests. The test results for the function have yet to complete, however (still “running” after five minutes – I looped back round and achieved 100%)

In T358589#9678132, @cmassaro wrote:

My two cents:

I would be careful about us doing too much type-checking in the frontend.

In T361654#9688889, @cmassaro wrote:

@GrounderUK , are you still seeing this behavior?

Please see https://www.wikifunctions.org/wiki/Wikifunctions:Project_chat#Question_regarding_identifying_failing_or_unconnected_tests. If that discussion goes no further, my feeling is that a reason for disconnection should always be given (whether it is a Test or an Implementation that is being disconnected).

In T282062#7074909, @Jdforrester-WMF wrote:

Is this properly a property of the Key or of the Type which uses it?

Yes. And no.
Please see my opinion at https://www.wikifunctions.org/wiki/Wikifunctions_talk:Representing_identity#Proposal_6:_Multi-typed_lists

We should include the implementation fail ratio for each test (simply the number of failing implementations over the total number of implementations, perhaps also expressed as the result of the division).
If we could also identify tests that had previously passed but currently fail, that would be especially interesting… maybe that would be a diff between successive reports or between the current view and a prior baseline? (So we could have a latest date for each test/implementation combination, by test, implementation and/or function… linked to the metadata — oh, just what we should have on the function page, really!)

Hint: as far as I can tell, the problem occurs only in Testers (for many functions) not in straightforward evaluations.

May help with T359772 (but that also mentions descriptions)

I’ve added a paragraph at https://www.wikifunctions.org/wiki/Wikifunctions:Naming_conventions#Search_characters and linked to that from https://www.wikifunctions.org/wiki/Wikifunctions_talk:Best_practices#Wikifunctions:Naming_conventions, linking to this ticket.

Thank you. I’ll document this somewhere on-wiki (any ideas where?) and then we can close this.

I encountered this error trying (lazily) to join a Natural number to a string. It disappeared when I converted the Natural number to a string first.

Please publish this intention on https://www.wikifunctions.org/wiki/Wikifunctions:Status to save people reporting bugs in the mobile interface.

I think we remembered to do this. See https://www.wikifunctions.org/view/en/Z14195.

Is there a correct way for me to suggest that “Verify in production” partially failed?
@Jdforrester-WMF (please see comments above)

In T360001#9634127, @AAlhazwani-WMF wrote:

that could work! i would also like to explore how might we include the zid directly to about widget, rather than the mediawiki header.

In T335583#9632190, @Amire80 wrote:

…

Can they be changed into "label" and "short description", or better, just "description"?

In T359802#9631817, @Jdforrester-WMF wrote:

When you run the tests during the edit, those values are uncached (as the text isn't saved). This is expected behaviour, but maybe we could mention that it's not cached for the very few people that might notice?

Yeah, but we shouldn’t do only this. Most users will want to do this *without* switching to qqx (“see it; fix it!”).

Maybe enable right-click and mobile equivalent?

…and here:
https://www.wikifunctions.org/wiki/Z13398?action=edit&uselang=en