I've recently tried to add some tests to WikibaseLexeme extension that make some edit using API: https://gerrit.wikimedia.org/r/#/c/335448/. Those edits, being a code from Wikibase's "core" code, may trigger Echo notification if Echo is enabled.

On Jenkins setup where those tests are Echo is indeed enabled. Once I've uploaded the final version of this patch to Gerrit, Jenkins reported some Echo test failure, see e.g. https://integration.wikimedia.org/ci/job/mwext-testextension-hhvm-composer-jessie/60/console.

With quite lame debugging I managed to figure out that when MWEchoThankYouEditTest::testFirstEdit is run, EchoNotificationMapper actually gets two thank-you notifications (one for first edit, and one for tenth edit), instead of single thanks for the first edit notification.
I suspect those extra edits messing up there are those made in WikibaseLexeme tests. It is still not really clear to me how come those edit are marked as done by the user MWEchoThankYouEditTest uses. https://gerrit.wikimedia.org/r/#/c/335448/ does not even get test user from MediaWikiTestCase.