User:Thunderkatz/python
refpunct.py: This is a python script that fixes punctuation and spacing around references. Requires files in pywikipedia. Outputs result to command line, after which user can copy and past over old article text. Run with "python refpunct.py name of article" (can be redirected to output file with "python refpunct.py name of article >outfile" - though this might not work because of unicode/ascii conversion problems).
Notes: Program is GIGO: anything other than the correctly spelled, case sensitive name (except for the first character) of a non-redirect page will produce an error. Program is very possibly inefficient and very possible can't handle some cases I haven't thought of.
import wikipedia import sys, re site = wikipedia.getSite('en', 'rationalwiki') name = sys.argv[1] for n in range(2, len(sys.argv)): name += " " + sys.argv[n] page = wikipedia.Page(site, name) text = page.get() c = text.find('\n<references') while text.find('\n<ref') >= 0: x = text.find('\n<ref') if x < c or c < 0: text = text[:x] + text[x + 1:] c -= 1 else: break while text.find(' <ref') >= 0: x = text.find(' <ref') text = text[:x] + text[x + 1:] reg = '<ref\s*((name|group)\s*=\s*".*?"\s*)*/>' char = [',','.',';',':','?','!'] for y in char: found = True while found: x = text.find('</ref>' + y) if x != -1: c = text.rfind('<ref', 0, x) text = text[:c] + y + text[c:x + 6] + text[x + 7:] else: found = False #endwhile found = True while found: m = re.search(reg + '\\' + y, text) if m != None: temp = m.group() x = len(temp) c = text.find(temp) text = text[:c] + y + text[c:c + x - 1] + text[c + x:] else: found = False #endwhile #endfor print text #if you wish to replace the text automatically with a bot (and you have a bot set up to connect), replace the previous line with: #page.put(text, comment='Fixing ref punctuation', watchArticle = None, minorEdit = True)
htmlremover.py:This is a python script that removes extraneous html from google discussion pages (e.g. the leaked ZB, Fab Five, etc.). Running is done through "python htmlremover.py infile >outfile"
import sys from string import find from string import rfind from string import strip f = sys.argv[1] title = False with open(f) as f2: for l in f2: if title: print "<h3>" + l + "</h3>" title = False if strip(l) == '''<span id="thread_subject_site">''': title = True if find(strip(l), '''<input id="hdn_date" type=hidden value="''') != -1: print "<i>" print strip(l)[40:rfind(l, ">") - 3] + " -" print "</i>" if find(strip(l), '''<input id="hdn_author" type=hidden value="''') != -1: print "<b>" print strip(l)[42:find(l, "<") - 3] + ":" print "</b> <br />" if find(strip(l), '''<a name="msg_''') != -1: if find(l, '''<a class=qt href="?hide_quotes''') == -1: print l else: temp = l[:find(l, '''<a class=qt href="?hide_quotes''')] + l[find(l, '''class=qt>''') + 9:len(l) - 7] print temp print "===================" print "<br />"
I release this code with no copyright (a.k.a. feel free to use, modify, burn, etc.), and no express or implied guarantee of any kind.