Extension talk:Replace Text/Archive 2018: Difference between revisions

Archives
2008 2009 2010 to 2011

Heiya, I would be cool if one would be able to add an additional reason for the replacement action taken to be shown in the log than just "Text replace - ...". Being able to localise "Text replace" would be another small wish. Cheers --[[kgh]] 21:48, 15 February 2012 (UTC)Reply

Just a note, might be a bug: Using a pipe symbol | in the search field in conjunction with regular expressions leads to following error message: Es ist ein Datenbankfehler aufgetreten. Der Grund kann ein Programmierfehler sein. Die letzte Datenbankabfrage lautete: (SQL-Abfrage versteckt) aus der Funktion „ReplaceText::doSearchQuery“. Die Datenbank meldete den Fehler „1139: Got error 'empty (sub)expression' from regexp (localhost)“.

Since I have a lot of SMW queries using the pipe, I came accross this. Anyway, I can of course search for patterns leaving out the |

--Francishunger (talk) 23:54, 22 March 2012 (UTC)Reply

I suppose that's because the pipe isn't part of the MySQL regex syntax, see http://dev.mysql.com/doc/refman/5.1/en/regexp.html for details. EDIT: Should have read through that page ... the pipe symbol should be recognized. --Nakohdo (talk) 07:25, 23 March 2012 (UTC)Reply

I'd like to batch move pages in category 'Person' from MAIN namespace to custom namespace 'PERSON'.

My setup

• MW: 1.16.x
• MySQL: 5.1.41-3ubuntu12.10
• Replace Text: 0.9.3

My Replace Text settings:

• Original text: (.*)
• Replacement text: Person:$1
• [x] Use regular expressions
• Search in namespaces: [x] (Main)
• Replace only in category: Person
• [x] Replace text in page titles, when possible

As a result I get messed page titles, e.g. a page Alex will be renamed to Person:APerson:Person:lPerson:Person:ePerson:Person:xPerson:

When changing the original text to A(.*) and the replacement text to Person:A$1 everything works as expected, moving all pages starting with letter A to the new namespace. Is this a bug or did I miss something?

--Planetenxin (talk) 07:42, 7 May 2012 (UTC)Reply

I'm not sure you can move namespaces like this as namespaces are treated differently than normal page titles. See http://www.mediawiki.org/wiki/Manual:Using_custom_namespaces for further pointers. hth --Nakohdo (talk) 14:16, 7 May 2012 (UTC)Reply

I read that the extension can't replace all words immediately. So I decided to wait and go to bed. But when I checked 7 hours later, not a single word has been replaced.

What would be the problem? My website hasn't yet been known to anyone except me and very few people who I personally know, so the traffic is very small. Is this the source of the problem?

If this problem can't be solved, how can I stop and cancel this wiki job?

You can manually perform all jobs waiting in the jobs queue by running RunJobs.php (link for directions) --FreedomFighterSparrow (talk) 13:50, 12 June 2012 (UTC)Reply

Running a cron job regulary for /maintenance/runJobs.php --type replacetext. --Subfader (talk) 10:28, 16 June 2012 (UTC)Reply

I found that the type replacetext dosn't work for me. Instead replaceText works. However, it seems like that these jobs are not executed if I run the job queue without explicitly specifying this type. LambdaB (talk) 14:20, 3 January 2013 (UTC)Reply

Is there a way to undo changes? Or do a mass revert?

If not, can this be a request :D

Unfortunately, there's no way to do that. Maybe this should be filed under the need for a tool to do mass reverts of all the edits (or all the latest edits) by one user - there's no way to do that in general, and that would be useful in dealing with spam as well. Yaron Koren (talk) 01:43, 17 July 2012 (UTC)Reply

I should note, though, that this script might be helpful until there's a true solution. Yaron Koren (talk) 12:54, 17 July 2012 (UTC)Reply

I cannot get this to work. It keeps saying No pages were found containing the string "(whatever I searched for)". I know that the string I am searching for is on at least one page. --Zackmann08 (talk) 16:52, 27 July 2012 (UTC)Reply

Hi - does it literally say that for any search string you type in? And by any chance, are you putting quotes in the search string? You shouldn't do that. Yaron Koren (talk) 16:58, 27 July 2012 (UTC)Reply

If I search for the word hello (so no quotes), it says No pages were found containing the string "hello". --Zackmann08 (talk) 20:37, 27 July 2012 (UTC)Reply

I assume that you have at least one page containing "hello"? Note also that the search is case-sensitive. What about just searching for a single letter? Yaron Koren (talk) 20:45, 27 July 2012 (UTC)Reply

When I search for a single letter it works. --Zackmann08 (talk) 23:00, 28 July 2012 (UTC)Reply

I can't get this to work either .. when I search for a word using the normal search I get like 30 pages containing the word (file name inside the pages), when I try to use the replace text to find this exact word and replace it I get the No pages were found containing the string ZJeFo 01:44, 11 Dec 2012 (KSA)

Are you looking in the right namespace(s), and is the capitalization exactly correct? Yaron Koren (talk) 05:09, 11 December 2012 (UTC)Reply

Yes I am, as I mentioned above, if I use normal search I get more than 30 results of what I am looking for .. If I use the same word I was searching for in the replace text extension I get No pages were found containing the string ZJeFo 20:57, 17 Dec 2012 (KSA)

As before, I don't know how much I can really help, with this set of information. Does the Replace Text search ever work? And if so, can you figure out what's different between when it works and when it fails? Yaron Koren (talk) 00:21, 18 December 2012 (UTC)Reply

Since this expansion already allows to select pages in very nice ways, I would love to mis-use it to append some text to the page instead of replace it.

You could just rule if "Original text" is empty, the "Replacement text" will be appended to all pages.--VolkoV (talk) 10:17, 6 August 2012 (UTC)Reply

You could also use a hook to display text below eahc articles. --Subfader (talk) 11:20, 6 August 2012 (UTC)Reply

I know, but if I want to change a lot of existing articles, this won't help.--VolkoV (talk) 11:27, 6 August 2012 (UTC)Reply

Empty "Original text" isn't allowed, it doesn't work. I want to add categories to all new files without a category. New uploaded files are empty, so it is impossible to replace something. Or is there another extension like add something to selected articles? Couldn't find something usable. Thanks a lot. --Tom Jac (talk) 07:22, 4 September 2013 (UTC)Reply

I'm glad an empty original text doesn't work, because that seems like it would be a recipe for disaster... you could try using regular expressions, and have a search text of ".*" and a replacement text of "$1 ABC"... that too might be a recipe for disaster, but maybe it would work. Yaron Koren (talk) 13:19, 4 September 2013 (UTC)Reply

Dangerous... But it finds all sites. After a careful manual selection it works. But is it possible to eliminate the non empty sites with a specific text (f.i. "[[category:") with a regular expression before? ".{0}" doesn't work... Sorry I'm not very familiarly with regular expressions. Thanks a lot --Tom Jac (talk) 17:39, 4 September 2013 (UTC)Reply

Sure - you could probably do something like "(.*Category:AAA.*)" for the search text. Yaron Koren (talk) 19:18, 4 September 2013 (UTC)Reply

Sorry (for my stupidity) but I couldn't get it. What is the search expression for something what is _not_ inside the site? All old sites have "Category:". So I could select all empty new ones. Thanks a lot. --Tom Jac (talk) 14:33, 17 September 2013 (UTC)Reply

Oh, oops - I misunderstood your earlier question. Unfortunately, I can't think of a way to do that... Yaron Koren (talk) 15:05, 17 September 2013 (UTC)Reply

How about "?!" for "not followed by" f.i. ".*(\[\[Category\:(?!Picture)\]\])" (perl, 'negative look-ahead assertion') for all sites without the category "Picture"? But it doesn't work. Thanks a lot --194.94.134.90 15:48, 15 October 2013 (UTC)Reply

I'm not seeing any "Use regular expressions" on the ReplaceText page. Is there some file that needs to be configured for this option to appear? Or is it just an outdated version of the extension... Hylian King ZW (talk) 13:48, 17 August 2012 (UTC)Reply

I assume it's the latter. Looking at the version history, it looks like that feature was added in version 0.9. Yaron Koren (talk) 14:05, 17 August 2012 (UTC)Reply

Alright, thanks for humoring a new sysop who is still fairly clueless about the inner workings of MediaWiki. Hylian King ZW (talk) 14:40, 17 August 2012 (UTC)Reply

How can i make it, that the Edits of this extensions was make vom from a Bot and not from the User who gives the Task to Start? 178.2.136.195 06:48, 8 November 2012 (UTC)Reply

Unfortunately, it's not possible to have a bot do it, because Replace Text doesn't offer an API action. But I think you can make it appear that a bot did it, by logging in as that bot and doing the replacement. Yaron Koren (talk) 13:40, 8 November 2012 (UTC)Reply

I hardcoded that into the extsnion cos I also think it's more useful this way. Maybe think of implenting a variable setting for it like $wgReplaceTextUser. --Subfader (talk) 14:46, 16 November 2012 (UTC)Reply

I agree. When I do a large replace operation on 10,000+ pages, it dominates the recent changes for days or weeks. Right now, I'm just logging in as a bot user to do the replace operation as Yaron suggested, but it would be handy if ReplaceText could be assigned a specific user in some cases. I don't want to do that in every case, so instead of a variable setting, it would be better if a check box could be selected to indicate the replace should be done as another bot user. Badon (talk) 23:21, 22 January 2013 (UTC)Reply

The bot account concept works fine for larger jobs, and keeps the edits to one user and with the bot flag, RC is not effected. Mlpearc (powwow) 18:50, 8 April 2013 (UTC)Reply

I run maintenance/runJobs.php --type replacetext regulary but when I set $wgCacheDirectory I get the error

Unable to open DB file for write "/tmp/cacheMsg/l10n_cache-en.cdb"
Backtrace:
#0 includes/Cdb.php(62): CdbWriter_DBA->__construct('/tmp/cacheMsg/l...')
#1 includes/LocalisationCache.php(883): CdbWriter::open('/tmp/cacheMsg/l...')
#2 includes/LocalisationCache.php(613): LCStore_CDB->startWrite('en')
#3 includes/LocalisationCache.php(352): LocalisationCache->recache('en')
#4 includes/LocalisationCache.php(262): LocalisationCache->initLanguage('en')
#5 includes/LocalisationCache.php(205): LocalisationCache->loadItem('en', 'magicWords')
#6 languages/Language.php(1946): LocalisationCache->getItem('en', 'magicWords')
#7 languages/Language.php(1960): Language->getMagicWords()
#8 [internal function]: Language->getMagic(Object(MagicWord))
#9 includes/StubObject.php(58): call_user_func_array(Array, Array)
#10 includes/StubObject.php(121): StubObject->_call('getMagic', Array)
#11 [internal function]: StubContLang->__call('getMagic', Array)
#12 includes/MagicWord.php(259): StubContLang->getMagic(Object(MagicWord))
#13 includes/MagicWord.php(197): MagicWord->load('rmatch')
#14 includes/parser/Parser.php(4289): MagicWord::get('rmatch')
#15 extensions/RegexFunctions/RegexFunctions.php(49): Parser->setFunctionHook('rmatch', Array)
#16 [internal function]: wfRegexFunctions()
#17 includes/Setup.php(370): call_user_func('wfRegexFunction...')
#18 maintenance/doMaintenance.php(86): require_once('/var/www/vhosts...')
#19 maintenance/runJobs.php(103): require_once('/var/www/vhosts...')
#20 {main}

Can this be fixed? MW 1.16a and ReplaceText 0.9.1 here but that shouldn't matter. Cheers. --Subfader (talk) 13:34, 24 November 2012 (UTC)Reply

I have no idea. Does this happen only for Replace Text jobs, do you know? Yaron Koren (talk) 16:03, 25 November 2012 (UTC)Reply

You're right, not only for RT. Wrong place here. https://bugzilla.wikimedia.org/show_bug.cgi?id=42437 --Subfader (talk) 20:59, 25 November 2012 (UTC)Reply

I get the following error when accessing the special page

Fatal error: Call to undefined method Xml::hidden() in C:\xampp\htdocs\wiki\extensions\ReplaceText\SpecialReplaceText.php on line 228

MediaWiki 1.18.1
PHP 5.3.8 (apache2handler)
MySQL 5.5.16

Luke 28 November 2012

I think you're just using an old version of Replace Text. Yaron Koren (talk) 14:26, 28 November 2012 (UTC)Reply

You appear to be correct, i downloaded what i thought was the latest from "download snapshot" - I downloaded the latest SpecialReplaceText.php from the git and the problem is resolved. Many Thanks - Luke 28 November 2012

Hi!

It doesn't seem to work inside Page Schemas' XML in category pages. Category pages containing search string don't come up at all in search results if the string is inside Page Schemas' XML. Other pages with the same string show up normally. Also if the search string is not inside XML, category page shows up normally in search results.

Feature request 1: I know that categories are irremovable by nature because of the inability of MediaWiki to replace all references to them inside pages content. But since Replace Text is the tool that does exactly that, wouldn't it be nice to be able to rename categories too somehow? i.e. by deleting them and copy their content to new ones? Even with extra warnings like "All the history of this caterory will be lost"?

Feature request 2 (Easy one, I might try to do it myself when I fully understand the development process): I would like to see replacement string as well in search results. Something like this (search string: less, replacement string: more):

...imagination is lessmore important than knowledge...

or even better

...imagination is lessmore important than knowledge...

--Ioannis Protonotarios ^{MW 1.19.3│PHP 5.3.19│MySQL 5.1.66│SMW 1.7.1│SRF 1.7.1│SF 2.5.2 alpha│SFI 0.6│PS 0.3.1│RT 0.9.5} 18:11, 31 December 2012 (UTC) Happy New Year!

Is there a hope for being able in the future to filter the pages to target not only by categories, but also by Concepts for people who use semantic mediawiki ?

I doubt it... that would only be helpful in a small set of circumstances, I think. One thing you can do, if you want to simulate that behavior, is to temporarily add the pages that would be in that concept to a new category, using #ifeq to match on the relevant properties or template fields. Yaron Koren (talk) 14:11, 21 January 2013 (UTC)Reply

Very nice and useful extension! Used it lots of times but for some reason it doesn't work for Russian characters (for example here ru.wikioffuture.org) If I search any word containing Russian characters (like награда) nothing will be found. Any advice would be greatly appreciated! --79.116.9.48 15:13, 16 May 2013 (UTC)Reply

Thanks! That's too bad - my guess is that it's the SQL query that's failing. Could the issue be with the MySQL encoding? Yaron Koren (talk) 17:38, 16 May 2013 (UTC)Reply

Thanks so much for your prompt reply Yaron! The DB character set is "utf8 -- UTF-8 Unicode" and collation is "utf8_general_ci". Should I change it :)

Hm - I don't know; utf8 sounds right. Sorry, I really don't know much about this encoding/non-ASCII thing. Yaron Koren (talk) 02:12, 17 May 2013 (UTC)Reply

Hi, there is the following situation. Trying to replace <ys4w /> to {{Соцссылки}}. All pages found and selected. But replacement happened only for 3 pages. Any ideas? Ektip (talk) 10:05, 1 June 2013 (UTC)Reply

That's strange. Maybe the rest of the replacement "jobs" just haven't run yet? Yaron Koren (talk) 15:38, 2 June 2013 (UTC)Reply

I had the same idea, and was waitng for several minutes, but it does not work. It also does not depend on what characters i replace Latin or Cyrilic. I did several iterations to replace the text of 40 pages. Sometimes it repaced on 4-5 pages, sometimes only for 1 page. When 8 pages left script did not replece any more. Ektip (talk) 06:14, 4 June 2013 (UTC)Reply

I want to change a lot of articles wich use the template "men". Here is one example for the call of template "men":

{{men
 |name=Schuster
 |age=24years
}}

After Replacetext this should be the result:

{{person
 |name=Schuster
 |firstname=
 |age=24years
}}

In Replacetext I marked "regular expression" and searched for:

{{men
 |name=(.*)
 |age=(.*)
}}

... and wanted to replace with:

{{person
 |name=$1
 |firstname=
 |age=$2
}}

Sadly it's not working :-( --87.186.34.168 20:26, 14 September 2013 (UTC)Reply

I am sorry ... I tried it another time and now it works :-) I don't not why. I don't delete this topic - maybe someone else needs the solution. --87.186.34.168 20:30, 14 September 2013 (UTC)Reply

Usage of this extension has been known to pose SEVERE SECURITY RISKS on sites that only use the default settings. Security measures for special page extensions should be used to prevent unauthorized usage which may lead to site and or server hijacking. The extension has vulnerabilities in various areas, including the usage of globals, PHP and MySQL database classes.

Note: Typical passwords will not prevent a breach against your site(s) if this extension is used.

It is strongly suggested that users of the extension refrain until an official version resolving the vulnerability is released; if usage is continued without resolution, then editing variables and re-declaring functions is suggested. Do not leave extension exposed on special page, if so, any user on the site will be able to change bulk content. Do not use if local settings have been leaked online; you may need to re-install the entire MediaWiki application if so. --Habatchii (talk) 02:59, 1 October 2013 (UTC)Reply

That sounds serious indeed. For what it's worth, by default only administrators have access to Special:ReplaceText, which I would think would lessen the risk considerably. In any case, do you know of specific text that can be entered into the "replace text" form that would cause problems? Feel free to respond either here, or by emailing me at yaron57 -at- gmail.com. Yaron Koren (talk) 03:49, 1 October 2013 (UTC)Reply

With all due respects, the extension is a valuable addition to MediaWiki; but my installation(s) have been under attack while using it. Apparently; the issue is URL related and suspected SQL injection directly into the string being parsed. The hack must have gained access and either changed specific text/strings or created a function to override the string whenever it was executed.

No rollback or previous information is ever recorded in the recent changes report. The first version of the works replaced are still in the searches and some in all pages, but the links lead to the affected pages. History information on such pages is empty, if the pages existed after the replace.

Recovery and downtime has increased tremendously since the attacks which appears to be targeting namespace variables. My suggestion would be to include a non-global option that can isolate URL/SQL execution as well as 'undoing by reason' to trigger an auto-debug report. Comments?

--Habatchii (talk) 07:02, 1 October 2013 (UTC)Reply

How do you know that these attacks are being done via Replace Text? Yaron Koren (talk) 12:15, 1 October 2013 (UTC)Reply

I have tested a locked patch (wiki) on a single server and minimized all other possible security threats. This extension is the single point of contact between the test negative and test positive output; no other extension was disabled during the safe period and enabled during the attack. It certainly was not a mechanical error of the parser, it was a genuine vulnerability attack.

--Habatchii (talk) 21:06, 1 October 2013 (UTC)Reply

Is Special:ReplaceText restricted to admins on your wiki? If so, how are hackers able to use it? Also, the way you phrase things makes it a little difficult to understand. What are "namespace variables"? What do you mean by "The first version of the works replaced"? What's an "auto-debug report"? Yaron Koren (talk) 21:22, 1 October 2013 (UTC)Reply

The extension was down (disabled) for a week. I brought it online for 2 minutes to change 4 pages and up to 10 other pages (template pages with links) were affected.(Note: It is a dummy test site so no other users are registered. Dummies host possible SQL and other vulnerable extensions without affecting the larger networks.)

Namespace variables are the individual parser variables that maintain a site's (wiki) namespaces. A list of such variables may be located on the official MediaWiki website.

First version refers to text before being replaced. Your extension has volumes of references on how the replace text feature works.

This notice is only to alert the original author of the extension that the extension has a major security issue that may affect corporate and or government clients that use it.

In evaluating this case, it appears that the hack is an organized group. Their objectives seem to be narrow at present but security issues are on their agenda. I would strongly encourage a peer review of this extension to reinforce the existing security already installed on the MediaWiki release and any third party tracking that may develop when in use. Perhaps tracing per users and reporting any suspicious IP address occurrences that do not match the authorized IP addresses.

--Habatchii (talk) 01:48, 2 October 2013 (UTC)Reply

Thanks for the info. So let me see if I understand: you ran Replace Text in order to change text in 4 pages, but after you ran it 10 other pages also had their text changed - and you believe that hackers were responsible for the changes to those 10 other pages? If that's correct, did the hackers do anything else, or was it just those 10 pages? Yaron Koren (talk) 03:00, 2 October 2013 (UTC)Reply

I will be launching a similar extension using the mainframe provided by the MediaWiki API. As a footnote to this issue, there are still extensible vulnerabilities with the Replace Text extension. I would also suggest that you change the name to a lessor profile name (i.e: PageGator or TextGator, etc.) Regardless if you have already implemented the suggested changes and or are currently monitoring usage statistics, you should be seriously prepared for peer review and general moderation of all aforementioned issues.

--Habatchii (talk) 04:50, 2 October 2013 (UTC)Reply

Well, again, I don't really understand what you're saying. If you ever feel like talking about this further, especially providing clearer information on the symptoms you've seen, that would be appreciated. Also, good luck with your extension. Yaron Koren (talk) 12:38, 2 October 2013 (UTC)Reply

Mediawiki: 1.23alpha php: 5.5.6

I installed extensions, enter special pages and tried to replace a simple word with another one. The extension found it on the page and said it's done but when I enter to the changed page, Nothing changed. — Preceding unsigned comment added by Omidh (talk • contribs)

I assume it's due to jobs; see the first item here. Yaron Koren (talk) 13:41, 29 November 2013 (UTC)Reply

I see the same issue. ReplaceText is a curent git checkout (December 10th 2013), MediaWiki is 1.22. I used basically the same checkout with MediaWiki 1.21 before and it worked correctly there. (The only changes in ReplaceText are some changed translations.)

When I click the "Do the replacement" button, then the jobs are not executed. I checked the database table "jobs" and I see the jobs from ReplaceText there. Their jobs_attempts field is 0, meaning MediaWiki did not try a single time to execute them and they also are not executed over time. And now I am clueless. What to do? --88.130.94.193 15:02, 15 December 2013 (UTC)Reply

Did you try running runJobs.php? I don't know why sometimes jobs aren't executed - it might be a memory issue. Yaron Koren (talk) 15:09, 15 December 2013 (UTC)Reply

Executing runJobs.php does run the jobs; the problem is not specific to Replace Text. However, it is reproducable for me that during normal page requests jobs are just not executed. That still worked correctly in MW 1.21. --88.130.94.193 15:16, 15 December 2013 (UTC)Reply

That has actually happened to me too, with MW 1.22... somehow I never made the connection to the MediaWiki version. Maybe this is a known bug in MW 1.22... if so, I hope it'll get fixed in time for the next version. I just added a note about it to the "Known issues" section in the documentation. Yaron Koren (talk) 03:26, 17 December 2013 (UTC)Reply

I filed that as https://bugzilla.wikimedia.org/show_bug.cgi?id=58719. Ideas are welcome! --88.130.81.165 01:11, 20 December 2013 (UTC)Reply

I'm having the opposite problem from the others with job issues. In my case, the replaceText jobs are run automatically by MediaWiki when I set $wgJobRunRate to a non-zero value, as expected. However, I prefer to use runJobs.php for performance reasons. On my server, runJobs.php hangs when it gets to the first replaceText job. It pops the job off the queue, as evidenced by jobs_attempts increasing to 1, but the script then hangs. Turning on profiling and the debug log shows that the two SQL calls by runJobs to pop the job occur, but nothing happens after that.

My setup is MW 1.22.0 with Replace Text's head revision, running on PHP 5.5, IIS 8.5, and Windows Server 2012 R2 against MySQL 5.6. If it's helpful, I have an older MW (1.20.2) installation on basically the same stack that I can use to try to isolate the issue if desired. --69.174.87.60 21:24, 20 January 2014 (UTC)Reply

That's strange. Sure, could you try doing the same thing on the MW 1.20 installation? Yaron Koren (talk) 21:29, 20 January 2014 (UTC)Reply

1.20.2 doesn't exhibit the same behavior -- it works as expected (both runJobs.php and the normal job runs on page loads work fine). The only difference between the two wikis in terms of extensions is that the 1.20.2 wiki has the Extension:Vector loaded (obsolete in 1.22), and the 1.22.0 wiki has Extension:PDF Handler and Extension:PDFEmbed loaded. --66.162.23.2 20:51, 24 January 2014 (UTC)Reply