RationalWiki:2013 Dead Links Project

Wiki-Project: Dead Links (sticky)

David noted that he'd been seeing dead links in the articles he was referring to on RationalWiki.RationalBlogs.org, so I put together a project page offsite that lists them all page by page. There may be some false positives because my tools are less forgiving that a browser. There's a raw dump of all dead links at the end of the list. Check it out. Please email me which pages you've cleared dead links from and I'll remove them from the list. If enough people actually do anything about this, I'll write an app that will let you check pages off the list yourselves. Surprise me. 03:47, 3 April 2013 (UTC)

So going by my quick skim of a few links, it seems that you're picking up some paywalled journal articles as broken links. Nebuchadnezzar (talk) 04:03, 3 April 2013 (UTC)

I briefly looked into this and I think some paywall services are sending out 404 status codes probably as a way to curb bot activity in general. Tmtoulouse (talk) 04:06, 3 April 2013 (UTC)

Nothing I can do about it without checking them by hand. Have at it! 04:11, 3 April 2013 (UTC)

I'll start working on it this weekend. Radioactive afikomen ^{Please ignore all my awful pre-2014 comments.} 05:11, 3 April 2013 (UTC)

I'm working up from the bottom of that list. Sophie^Wilder 12:00, 3 April 2013 (UTC)

As stuff frequently comes and goes on the internet should we <capture> some of these external links? Генгис^mutating 12:50, 3 April 2013 (UTC)

Since it's not a live list, it sounds like "signing up" would be helpful, so I'll start with the laundry Ball section. Godot The ablity to breath is such an overrated ability 13:09, 3 April 2013 (UTC)

May be an idea. Including archive.org substitute links - David Gerard (talk) 16:43, 3 April 2013 (UTC)

Sorry for letting "this weekend" turn into "six weeks from now." I've been busy with school. But I'm on it now! You can clear Louie Gohmert from your list. (Another reason I've been avoiding this task: I often get distracted from cleaning up problems spread across a range of articles by trying to majorly revamp every page, and it feels like I'm not making any progress. The Gohmert article became a three-hour rewrite.) Radioactive afikomen ^{Please ignore all my awful pre-2014 comments.} 04:29, 26 May 2013 (UTC)

What is going on here?

So, basically 2 people emailed me to help me keep track of making sure this list is current while they work through it. 2 people. David Gerard gave people who care a reason to do something about our dead links - he's generating a lot of traffic promoting RW and the RWF on his RW blog - it's been a resounding success in the month or so it's been online. I'm trying to start a dialog about quality of content in which so few people appear interested in participating that it's not going to go anywhere. I don't know what to do. 17:10, 4 April 2013 (UTC)

I didn't figure you wanted a blow by blow email, but something at the end of a large set of changes. Makes it easier on everyone to wait till you've done 5 or 10 on a list. It's not exactly hard to figure out when changes have been made in teh short term.Godot The ablity to breath is such an overrated ability 17:13, 4 April 2013 (UTC)

I'll pitch in, but I'm unsure how to proceed. I take it we're supposed to find a new source rather than simply delete the link? --TheLateGatsby (The end of the dock ) 17:16, 4 April 2013 (UTC)

Personally, I'd say each link is different, and there is no "one way" to proceed. A new source is usually the best, but sometimes the links are the point of the comment, and the comment should be changed. Sometimes the links are redundant and can just be deleted. Godot The ablity to breath is such an overrated ability 17:20, 4 April 2013 (UTC)

Alright, I'll do a few in between bouts of work. Sorry I didn't notice this, Nutty. Did he doubt/Or did he try? 17:23, 4 April 2013 (UTC)

Does the list auto-update? I don't want to undo work that's already been done. Did he doubt/Or did he try? 17:24, 4 April 2013 (UTC)

It does not. Nutty has to manually update the pages, which is why he wants email updates on what's been changed. I am working on "laundry balls through music" for what it's worth.Godot The ablity to breath is such an overrated ability 17:28, 4 April 2013 (UTC)

I don't think I'll need to write an app to let people check pages off themselves if they're clearing them out in blocks and let me know as they go. I've got my laptop with me 24/7 so it's easy enough to update pages pretty quickly so people do find themselves frustrated trying to duplicate efforts. Thanks. 23:26, 4 April 2013 (UTC)

I'm not in a hurry at all, Godot. I'm just curious why so few people are interested in helping with a simple, though perhaps time consuming project that is core to RW's very existence. The more people pitch in, the fast it goes. 17:38, 4 April 2013 (UTC)

I haven't been to the Saloon Bar much, recently. --TheLateGatsby (The end of the dock ) 17:42, 4 April 2013 (UTC)

A day and a half in seems a bit early to be complaining about lack of engagement for something time-consuming which was launched in the middle of the week. I also found the intercom message confusing: "cycles to spare" isn't something I've encountered before, & looked like jargon for maybe something a bot does. Anyhoo, I'll have a look at a few of these. Is collating progress offsite & via email really the best way? It would seem more convenient to put up a project page at RW where we can note which pages we've already done as we go along. ΨΣΔξΣΓΩΙÐ^{Methinks it is a Weasel} 17:47, 4 April 2013 (UTC)

I'm trying to remove pages as quickly as people advise they've fixed them, but sure a project page would be great. 23:24, 4 April 2013 (UTC)

(OT - you've never heard 'cycles to spare? huh. i use that phrase often). Godot The ablity to breath is such an overrated ability 17:50, 4 April 2013 (UTC)

I'll take Barry Goldwater through Core Issues Trust. --TheLateGatsby (The end of the dock ) 17:59, 4 April 2013 (UTC)

Thanks Gatsby. 23:26, 4 April 2013 (UTC)

Please advise of your progress every once in a while so people don't duplicate efforts - I'm usually updating the project pages within minutes of receiving notice. 17:54, 6 April 2013 (UTC)

I'm poking around the "L's", but mostly the ones with only 1-2 bad links. Did he doubt/Or did he try? 18:09, 4 April 2013 (UTC)

List updated 4/4/13

Ok thanks to the people working on this - uploaded updated pages a few minutes ago. Thanks y'all. 23:22, 4 April 2013 (UTC)

Again and added some nav help. 02:56, 5 April 2013 (UTC)

archive.org

Quite a lot of broken links are old crank pages that are nevertheless relevant. Old copies of these can often be found in archive.org.

So, nicked from Wikipedia: {{Template:Wayback}} - David Gerard (talk) 07:10, 5 April 2013 (UTC)

Excellent offering, David. The wayback machine is awesome. It's the last part of the internet not owned by ogle or microsquish. ħuman 03:03, 7 April 2013 (UTC)

Status

Approximately 450 links down. 4348 to go. Can someone sticky this thread please? 17:52, 6 April 2013 (UTC)

You mean the subsection? Pibot doesn't do subsections, just whole sections, so you should be ok. Meanwhile, this is quite a job. It took me an hour last night just to do Wolf hysteria. Sophie^Wilder 16:09, 7 April 2013 (UTC)

I didn't know the syntax for it at all - David got it for the section. I'll selectively archive some of this stuff as we get further along. Yes, it's shaping up to be a bigger deal than I thought. It takes longer than it seems like it might. Heh should I post a daily progress bar or would that be demoralizing? 16:15, 7 April 2013 (UTC)

Weekly? Or a number count on relevant people's talk pages? Sophie^Wilder 16:20, 7 April 2013 (UTC)

Daily. HOURLY. 16:27, 7 April 2013 (UTC)

Wow, you guys are doing great. This is really moving along. Glad to see so many people interested. </sarcasm> About half a dozen people are chipping away at these. It's not a very important project. 14:43, 10 April 2013 (UTC)

I would love to be able to help, and in a few weeks I will start to lend a hand.--^talk 15:00, 12 April 2013 (UTC)

Is this still going on? Can I still help?--^talk 23:43, 13 May 2013 (UTC)

Yes, please do! --TheLateGatsby (The end of the dock ) 23:59, 13 May 2013 (UTC)

Please. Email me what you fix and I'll remove it from the list ASAP. 00:31, 14 May 2013 (UTC)

Working links

Some of the links listed work fine for me, although they sometimes require a refresh. The ones I've noticed so far are for Jim Dowson and Kinoki Foot Pads. EDIT: Enoch Powell too. Balaam (talk) 16:58, 7 April 2013 (UTC)

Also Evo Morales, William Luther Pierce, VDARE, What the Bleep Do We Know? and William Nelson. Balaam (talk) 17:41, 7 April 2013 (UTC)

There are false positives because I'm using a programming library that strictly follows HTTP (unlike your browser), hates special characters, and times out quickly. Or it would have taken more than the ungodly number of hours it already took to check all the links in the first place. 21:31, 7 April 2013 (UTC)

Updated. Thanks Balaam. 21:44, 7 April 2013 (UTC)

No problem. A few more which can be removed from the list: 2012 U.S. Presidential Election, Abraham Lincoln, African National Congress, Agave nectar, Agnosticism, Aleister Crowley, Alveda King, American Civil War, Amish, Amphetamines, Anti-Catholicism and Antidepressant. Balaam (talk) 07:30, 8 April 2013 (UTC)

Thanks much. Updated. 16:10, 8 April 2013 (UTC)

Bethlehem, Birth control, Bisexuality, Bob Larson, Breeder reactor, CW Leonis, CMI list of scientists alive today who accept the biblical account of creation, California Proposition 8, Carl Sagan, Capalert, Clitoris, Chuck Norris, Cold War and Confirmed bachelor can go too. Balaam (talk) 16:32, 8 April 2013 (UTC)

Perhaps we should continue this thread in RWspace? Sophie^Wilder 17:44, 8 April 2013 (UTC)

So it's easier to ignore? What's the point in citing anything anyway. 14:51, 10 April 2013 (UTC)

Corporal punishment, Council for National Policy, Crystal healing, Daily Telegraph, Denmark, Dog whistle politics, Earth First!, Dungeons and Dragons, Egmond Codfried, Excommunication, Eusapia Palladino and Evangelical agenda are all false positives. Balaam (talk) 17:05, 10 April 2013 (UTC)

Recently fixed Franz Boas and psychohistory. Nebuchadnezzar (talk) 02:32, 11 April 2013 (UTC)

NY Times links that have "Query" in them tend to still work (they just redirect?) and NPR's links seem to always be false positives. --TheLateGatsby (The end of the dock ) 15:23, 8 July 2013 (UTC)

Thanks. I will purge them when I can get to it in the next few days. In the meantime please just click them off. 17:40, 8 July 2013 (UTC)

More corrected articles

Barry Goldwater (1 link, replaced); Bart Sibrel (Link is correct); Baruch Spinoza (1 link, deleted); Benjamin Franklin (1 link, replaced); Bob Barr (Found new quotes displaying his shifting position on AGW); Book burning (1 link, replaced); Charles Coughlin (1 link, replaced); Chick-Fil-A (1 link, deleted, as it could not be otherwise supported). --TheLateGatsby (The end of the dock ) 02:22, 12 April 2013 (UTC)

Bible Belt (1 Link, deleted -Deleted whole section); Big Placebo (1 Link, deleted); Blaming the victim (1 Link, replaced with mirror); Book of Genesis (1 Link, replaced); Bush Derangement Syndrome (1 Link, replaced) TheLateGatsby (The end of the dock ) 18:16, 15 April 2013 (UTC)

Thanks much. No need to say what you fixed in each article. 14:48, 17 April 2013 (UTC)

You're welcome. I know I've been slow, but I'll try to get a few per day. Is there a better place to put this? Should I email you directly? --TheLateGatsby (The end of the dock ) 14:55, 17 April 2013 (UTC)

Email makes this a lot easier. 15:13, 17 April 2013 (UTC)

Caching?

I think that at least part of the updated statistics are inaccurate due to squid caching - the lists are timestamped as updated today, but I just found an article that had had its links "fixed" (Privatization), but the list still contains the old links.--ZooGuard (talk) 15:22, 17 April 2013 (UTC)

it's not cachine. It's me mistakenly not removing that entry. It happens more often when people skip around the list. 15:28, 17 April 2013 (UTC)

Paywalls

What should we do regarding articles behind paywalls? --TheLateGatsby (The end of the dock ) 14:37, 25 April 2013 (UTC)

Depends on the situation. Leave them in, put a warning. Try finding another source and add it as an additional citation. If it's an academic article, it may be freely accessible via its original journal's page or the researcher(s) homepage.--ZooGuard (talk) 15:42, 25 April 2013 (UTC)

Quite a lot of paywalled journal articles are, nevertheless, the actual citation for the matter in question. The citation should stay, but maybe mark as paywall, I dunno (I tend not to bother) - David Gerard (talk) 22:43, 26 April 2013 (UTC)

At the risk of making too broad a statement, in the few quantum mechanics related articles I've looked at so far, quite a few paywall journals (including Nature) are hitting false positives with Nutty's app. While that information isn't super useful, it might make it a bit easier to handle science related articles. I thought it was strange to see a link to Nature on the DLP, since most reputable journals don't start archiving their articles (if they do so at all) for a good 30 years or so after publication. - GrantC (talk) 16:21, 17 September 2013 (UTC)

I wish I could do something about it, but the HTTP client I wrote is super sensitive. I don't want to hurt its feelings. I'm sure there are other false positives. I only pulled headers to see if there was a resource at that location. I guess some servers don't allow only header requests. Shrug. Pulling the whole page would have made the operation take orders of magnitude longer. 22:29, 24 September 2013 (UTC)

Yeah, it's not really something to worry about, I think, as it's pretty easy to quickly check these cases. My experience with most scientific journal servers is that they're bloated and complicated, so it doesn't surprise me that they would be kicking up false positives. I just figured I would point it out to make it a bit easier for folks to quickly skim and check some scientific articles that may otherwise have been avoided by anyone without institutional access to such journals. - GrantC (talk) 22:34, 24 September 2013 (UTC)

Just below 1,400

We now have fewer than 1,400 links left. Much respect to everyone who's been working on this! --TheLateGatsby (The end of the dock ) 16:03, 1 July 2013 (UTC)

Yay! Thanks for keeping track, but you'd be disappointed to know how many other people take quality on RW seriously. 16:34, 1 July 2013 (UTC)

Yeah... I'm trying to stay enthused, despite people's indifference. I'm really into this project because it's a way to improve the site without stepping on people's toes. It sucks so many people don't care, but at least no one's actually against this project. --TheLateGatsby (The end of the dock ) 16:56, 1 July 2013 (UTC)

1,300

Whoo! --TheLateGatsby (The end of the dock ) 17:49, 23 July 2013 (UTC)

1,200

Huzzah! It took a while, but that's another hundred down. Once we hit Zero, we should all get plastered before Nutty has to put up the next batch of dead links. --TheLateGatsby (The end of the dock ) 15:22, 5 August 2013 (UTC)

1,100

My apologies for my inactivity -obviously, we're not obligated to work on the DLP, but I do feel it's important. I'll have to change tack -rather than sporadically fix links, I'll try doing a bit more research and see if I can improve articles as I go through their references. More importantly, I'll try making some time to actually work on articles. As you can see from my contribs, I tend to just bounce around the list, and that has to stop. --TheLateGatsby (The end of the dock ) 15:12, 4 September 2013 (UTC)

1,000

Yay! --TheLateGatsby (The end of the dock ) 15:21, 25 October 2013 (UTC)

800

Holy crap, I'm not dead. --TheLateGatsby (The end of the dock ) 15:13, 3 June 2014 (UTC)

Nice, guys! When we get closer to the end, I'll spring a totally demoralizing surprise I've been keeping in reserve to totally demoralize you all. Nutty Roux (talk) 16:26, 3 June 2014 (UTC)

Heh. The joke's on you, morale couldn't be lower! Now, if you'll excuse me, I'm going to start some new stubs, then dare people to take us to court. --TheLateGatsby (The end of the dock ) 16:41, 3 June 2014 (UTC)

We're under 700. Can we have the surprise yet? Pleeease? --TheLateGatsby (The end of the dock ) 15:57, 5 June 2014 (UTC)

Surprise: Nutty's been reverting every single dead link change from the very beginning. Now you get to start over! - Grant (talk) 16:03, 5 June 2014 (UTC)

What a tweest! --TheLateGatsby (The end of the dock ) 16:25, 5 June 2014 (UTC)

The surprise is I accidentally left some out when I forgot to dump a file into the database. The next surprise is that I'll run the script again when I reseed the database and catch all the dead links that popped up in the last year. Nutty Roux (talk) 16:58, 5 June 2014 (UTC)

You son of a bitch. - Grant (talk) 17:12, 5 June 2014 (UTC)

No worries. --TheLateGatsby (The end of the dock ) 17:33, 5 June 2014 (UTC)