RationalWiki:Technical support
Herein goeth anything regarding the technicalities of the wiki
- Problems with accessing/editing the wiki.
- Problems with MediaWiki/HTML/CSS/Javascript/etc.
- Questions about RW's configuration/extensions/servers/etc. See gathered planned changes to-do items.
- In short, anything tech-y goes here.
External links
- Semi-frequent updates about RW's technical status are available at RationalWiki's blogspot.
- To view or suggest changes to configuration files and MediaWiki extensions, see RationalWiki's GitHub.
Users
- Users with Tech rights: see Special:ListUsers/tech.
- Users with root on the servers: David Gerard (dgerard@gmail.com), Tmtoulouse and Tim Starling.
If your IP is blocked from editing as an open proxy
- You need to go to your provider about it. The lists we use are xbl.spamhaus.org, dnsbl.tornevall.org, all.s5h.net. If your IP is on one of these lists, then your provider has a serious problem they need to resolve. We cannot get your ISP out of the blacklist.
The DDoser is back[edit]
See above. Up to 2 mins hang 2001:569:FDC9:AB00:AD36:DF77:E8E8:A4B1 (talk) 21:03, 5 July 2024 (UTC)
- Becoming a problem again. Whatever limiting measures were in place before are seemingly not sufficient. Plutocow (talk) 18:20, 13 August 2024 (UTC)
Looking at this, need to ascertain the incoming - David Gerard (talk) 11:56, 31 August 2024 (UTC)
- So far I haven't caught the asshole in the act, but when I do, oooh blockaroonie - David Gerard (talk) 13:44, 5 September 2024 (UTC)
Looking at daily biggest user IPs, and 503ing anything that seems to be a cloud hosting range. If this causes issues for a real user, please let me know. But basically automated access from a cloud IP range is a big red flag right now - David Gerard (talk) 11:36, 12 September 2024 (UTC)
- Thanks for dealing with this, David. Chef Moosolini’s Ristorante ItalianoMake a Reservation 12:48, 12 September 2024 (UTC)
FWIW, some of our most obnoxious effective DDOSers appear to be AI companies scraping the web without rate limiting (yes, I'm blocking them as I spot them) - David Gerard (talk) 19:15, 17 September 2024 (UTC)
Hundreds of RationalWiki articles deindexed by Google[edit]
Can this issue be fixed? Hundreds of articles have been deindexed including Kent Hovind, Jordan Peterson, Nigel Farage, Shawn Baker and most of the pseudoscientific racists (Emil Kirkegaard, John Fuerst, Edward Dutton, Anatoly Karlin, Bryan Pesta, Bo Winegard, etc).91.242.215.168 (talk) 14:52, 19 September 2024 (UTC)
- It's not particularly focused on articles on people, like you listed. I searched for more types of articles and it looks rather random what's missing, it's independent of what the topic is. For example, one article on a logical fallacy was missing on Google (only the talk page showing up) while another wasn't. --ApooftGnegiol (talk) 15:56, 19 September 2024 (UTC)
- Argumentum ad nauseam shows up. Carthage (talk) 16:06, 19 September 2024 (UTC)
- England and Scotland are missing too. Chef Moosolini’s Ristorante ItalianoMake a Reservation 16:10, 19 September 2024 (UTC)
- Argumentum ad nauseam shows up. Carthage (talk) 16:06, 19 September 2024 (UTC)
- [EC] Some articles on people also show up, they're not all gone. Our QAnon article is deindexed. A guess of mine is that the higher the traffic to a page, the greater the likelihood it was deindexed. Pages gone seem to include prominent ones disproportionately, but I could be off, I haven't checked hundreds. --ApooftGnegiol (talk) 16:13, 19 September 2024 (UTC)
- But only mainspace pages seem gone this way. Search for Conservapedia and you find the pages outside mainspace but not the main one. --ApooftGnegiol (talk) 16:22, 19 September 2024 (UTC)
- Suppose a place to start would be here. Chillpilled (talk) 18:38, 19 September 2024 (UTC)
- The main suggestions there for doing more require Google accounts related to representing a site. Now the question is who, if any, has such Google accounts for RW. (Trent? Anyone else?) I've now brought it up with the board. --ApooftGnegiol (talk) 10:30, 20 September 2024 (UTC)
- My guess is Gerard. Chillpilled (talk) 11:28, 20 September 2024 (UTC)
- This is very frustrating. The Paul Saladino article had massive traffic and was the top search on Google for his name and now it has also disappeared. A lot of people on social media were recognizing that article. The same with the Eric Berg article I created as it was the top search result for Berg for years. I am hoping this issue can be corrected and Google adds these links back to the top. Even Kent Hovind has been removed from Google. I am not sure why any of these articles have been de-indexed in the first place. I have no idea why a lot have been removed and some still exist or how to fix it. The site will be losing a lot of traffic because of this. Johns (talk) 15:11, 20 September 2024 (UTC)
- Many more RW articles have disappeared from Google. This wiki is practically now dead in terms of traffic. There used to be a lot of shared links on social media - no longer the case since people are not finding articles on websearch. 108.61.168.133 (talk) 12:33, 21 September 2024 (UTC)
- This is very frustrating. The Paul Saladino article had massive traffic and was the top search on Google for his name and now it has also disappeared. A lot of people on social media were recognizing that article. The same with the Eric Berg article I created as it was the top search result for Berg for years. I am hoping this issue can be corrected and Google adds these links back to the top. Even Kent Hovind has been removed from Google. I am not sure why any of these articles have been de-indexed in the first place. I have no idea why a lot have been removed and some still exist or how to fix it. The site will be losing a lot of traffic because of this. Johns (talk) 15:11, 20 September 2024 (UTC)
- My guess is Gerard. Chillpilled (talk) 11:28, 20 September 2024 (UTC)
- The main suggestions there for doing more require Google accounts related to representing a site. Now the question is who, if any, has such Google accounts for RW. (Trent? Anyone else?) I've now brought it up with the board. --ApooftGnegiol (talk) 10:30, 20 September 2024 (UTC)
- Suppose a place to start would be here. Chillpilled (talk) 18:38, 19 September 2024 (UTC)
- But only mainspace pages seem gone this way. Search for Conservapedia and you find the pages outside mainspace but not the main one. --ApooftGnegiol (talk) 16:22, 19 September 2024 (UTC)
Are we sure there was not some huge legal complaint to Google to deindex/derank articles? Recently Anatoly Karlin was complaining and may have sent something.
As I noted in the Aporia post (https://aporiamagazine.com/p/the-radical-left-website-trying-to), the real problem isn't with individual harassers or even @RationalWiki per se, but that Google privileges a website with EncyclopediaDramatica editorial standards as if it were Wikipedia on pagerank.
149.28.172.61 (talk) 15:50, 21 September 2024 (UTC)
- I have noticed that while RW results might have remained, they've lost the snippets - for example not even wrong has. Which is now the #7th suggestion. However, DuckDuckGo has us at #4 and Bing/Yahoo has us on #3. Brave Search has only talk page for it at about #25 [being beaten by frigging Urban Dictionary]? 'Project Blue Beam' still is at #2, but has lost it's snippet. 'Ethan Ralph' still is #1, but again no snippet.
- I then started to use a few SEO tools. We have [apparently] a domain authority of 75 and a page authority of of 58, which is better than CP and CZ but not by much [their numbers appear to be about a 'decent forum' level]. SEOs might think it's a 'vanity mechanic' but it at least reveals they think RW is better than a forum [but no way like a wiki; which normally is around 90/60 - even little ones like Bulbpedia and Elder Scrolls ones]. Another complained we don't have a sitemap [they also complained about the lack of metatags, but who the fuck still uses them in '24?]. The third one was more worrying.
- According to it's traffic measurer [how do they do this?] RW has crashed. '17-'18 RW was getting 500k a month, '19-'20 was 200k a month, by mid '21 we were at 40k a month. There it kicked along until mid-'23, in which it started to climb back to about 100k/month. Then in May this year we shed about 35% and we've been kicking around 80k/month since.
- It doesn't take a genius to come up with the theory that Google has depreciated RW, we are no longer showing up in lots of search results and thus, our viewership is crippled because of it. KarmaPolice (talk) 17:32, 21 September 2024 (UTC)
- The problem isn't pagerank (which I was in the slow process of improving by fixing broken links on pages but have paused doing so pending whether this is fixed). The site is largely delisted entirely. Someone will need to contact Google somehow. Chillpilled (talk) 19:22, 21 September 2024 (UTC)
- I would not be surprised if it is related to settlement of the recent legal dispute. I recall reading that the guy I will not name has been known to request the Internet Archive suppress pages, so it's likely he requests that Google removes them from Google Cache as well, and my guess is someone (or a team of people) at Google reviewed his request and possibly the site as a whole, decided RationalWiki should not be such a high-ranking website, and took one of those "manual actions" that they mention. Last I checked, BlueCoat Systems (a division of Norton) had downgraded the site from "Reference" (generally considered work safe) to "Mixed Content (Potentially Adult)" (generally considered NSFW, which is probably why I can't access RW from work or from the wi-fi at places like the Pecanland Mall in Monroe, LA) as well. I know no one wants to hear this, but this is further proof that this is, in fact, the beginning of the end for RW. 71.208.249.110 (talk) 21:05, 21 September 2024 (UTC)
- This has nothing to do with any legal dispute, it is an issue related to IPS and Google bot (see below). This isn't the beginning of an end, stop using this page to promote irrational conspiracy theories. There is nobody behind the scenes trying to take down RW by sending complaints. It's a temporal technical problem. Johns (talk) 02:01, 22 September 2024 (UTC)
- FYI, there was in fact a complaint to Google earlier this year, requesting that they remove RationalWiki from their search results. Information about the complaint is here. However, I don't know whether the current problem is related to that complaint or whether it's a coincidence. Tetrapteryx (talk) 07:48, 22 September 2024 (UTC)
- That seems to be about one specific article, now deindexed for the same technical reason as the rest. There's no evidence of Google people taking any steps to censor or downgrade our contents. I think they usually only take stuff down in response to requests with a formal legal justification (so never in response to a critic just saying "I think it's defamatory"). On page rank etc. more generally, we can in the future also use Search Console info to try to work better with Google's machinery. --ApooftGnegiol (talk) 10:05, 22 September 2024 (UTC)
- Even before the update below I found it unlikely Google would knowingly delist the website, since from what I've seen that seems to be a pretty rare/extreme measure for them to take, that they mainly do for legal reasons. Chillpilled (talk) 10:54, 22 September 2024 (UTC)
- The way libel complaints to deindex content from Google works - is the requests are only blocked in the country of where the sender resides. This means if Google did approve Anatoly Karlin's libel complaint to block his article it would only be blocked in the UK since he wrote he lived there. That's clearly not what happened since his article is deindexed outside of the UK and worldwide (in the US, France, Australia etc., I checked on VPNs). Secondly, you need court orders for Google to approve the libel complaint which Karlin does not have. Google does not block content said to be libel without court orders (I have checked this before). So Karlin's complaint was frivolous.207.148.81.51 (talk) 12:57, 22 September 2024 (UTC)
- Ps. It is by no means clear that Anatoly Karlin filed that legal complaint. Could someone have been impersonating him? I do not believe Karlin lives in the UK. There has been a malicious troll filing false legal complaints against RationalWiki while impersonating other people and those can be found on the Lumen Database. For example, in 2021 a libel complaint was filed for Jonathan Anomaly's page also from the UK (you can easily identify this based on "Tulane University" and "advocating liberal eugenics".) Anomaly does not live in the UK either. This also looks like a false DMCA in Anomaly's name (some BONs were spamming a link to it on various talk pages).207.148.81.51 (talk) 13:13, 22 September 2024 (UTC)
- Good on Google for having someone actually reviewing these manually, evidently. Chillpilled (talk) 13:19, 22 September 2024 (UTC)
- Ps. It is by no means clear that Anatoly Karlin filed that legal complaint. Could someone have been impersonating him? I do not believe Karlin lives in the UK. There has been a malicious troll filing false legal complaints against RationalWiki while impersonating other people and those can be found on the Lumen Database. For example, in 2021 a libel complaint was filed for Jonathan Anomaly's page also from the UK (you can easily identify this based on "Tulane University" and "advocating liberal eugenics".) Anomaly does not live in the UK either. This also looks like a false DMCA in Anomaly's name (some BONs were spamming a link to it on various talk pages).207.148.81.51 (talk) 13:13, 22 September 2024 (UTC)
- The way libel complaints to deindex content from Google works - is the requests are only blocked in the country of where the sender resides. This means if Google did approve Anatoly Karlin's libel complaint to block his article it would only be blocked in the UK since he wrote he lived there. That's clearly not what happened since his article is deindexed outside of the UK and worldwide (in the US, France, Australia etc., I checked on VPNs). Secondly, you need court orders for Google to approve the libel complaint which Karlin does not have. Google does not block content said to be libel without court orders (I have checked this before). So Karlin's complaint was frivolous.207.148.81.51 (talk) 12:57, 22 September 2024 (UTC)
- Even before the update below I found it unlikely Google would knowingly delist the website, since from what I've seen that seems to be a pretty rare/extreme measure for them to take, that they mainly do for legal reasons. Chillpilled (talk) 10:54, 22 September 2024 (UTC)
- That seems to be about one specific article, now deindexed for the same technical reason as the rest. There's no evidence of Google people taking any steps to censor or downgrade our contents. I think they usually only take stuff down in response to requests with a formal legal justification (so never in response to a critic just saying "I think it's defamatory"). On page rank etc. more generally, we can in the future also use Search Console info to try to work better with Google's machinery. --ApooftGnegiol (talk) 10:05, 22 September 2024 (UTC)
- FYI, there was in fact a complaint to Google earlier this year, requesting that they remove RationalWiki from their search results. Information about the complaint is here. However, I don't know whether the current problem is related to that complaint or whether it's a coincidence. Tetrapteryx (talk) 07:48, 22 September 2024 (UTC)
- This has nothing to do with any legal dispute, it is an issue related to IPS and Google bot (see below). This isn't the beginning of an end, stop using this page to promote irrational conspiracy theories. There is nobody behind the scenes trying to take down RW by sending complaints. It's a temporal technical problem. Johns (talk) 02:01, 22 September 2024 (UTC)
- I would not be surprised if it is related to settlement of the recent legal dispute. I recall reading that the guy I will not name has been known to request the Internet Archive suppress pages, so it's likely he requests that Google removes them from Google Cache as well, and my guess is someone (or a team of people) at Google reviewed his request and possibly the site as a whole, decided RationalWiki should not be such a high-ranking website, and took one of those "manual actions" that they mention. Last I checked, BlueCoat Systems (a division of Norton) had downgraded the site from "Reference" (generally considered work safe) to "Mixed Content (Potentially Adult)" (generally considered NSFW, which is probably why I can't access RW from work or from the wi-fi at places like the Pecanland Mall in Monroe, LA) as well. I know no one wants to hear this, but this is further proof that this is, in fact, the beginning of the end for RW. 71.208.249.110 (talk) 21:05, 21 September 2024 (UTC)
- The problem isn't pagerank (which I was in the slow process of improving by fixing broken links on pages but have paused doing so pending whether this is fixed). The site is largely delisted entirely. Someone will need to contact Google somehow. Chillpilled (talk) 19:22, 21 September 2024 (UTC)
I'll agree that Karlin's complaint is unlikely to have caused this, but I also think it's unlikely to be a purely technical problem, for three reasons. First, If RationalWiki's safeguards to prevent DDoS attacks were accidentally blocking Google's bots, as suggested below, that would have affected all types of pages across the site. But the deindexing has applied only to pages in article space, while talk pages and project pages seem unaffected, which is more in line with what I'd expect if this were an intentional action by Google.
Second, I'm familiar with how a page's rank is affected when Google bots are prevented from accessing it, based on observing that on domains I've owned in the past. When that happens its Google rank steadily declines over a period of two to three weeks before it disappears from the results. But what's happened here, in every case that I've watched, is that RationalWiki's article was in the top five results for its subject, and then the following day the article had disappeared from Google. Google bots being prevented from accessing a page doesn't normally cause it to disappear from search results that abruptly.
Lastly, as KarmaPolice mentioned, RationalWiki's rank in Bing and DuckDuckGo has been unaffected. There aren't many configuration errors that could make pages inaccessible to Google and no other search engines, unless you deliberately configured the site to exclude bots only from Google. But this outcome is consistent with results of searches for articles at Infogalactic, which has had most of its pages deprecated by Google but not by other search engines. Compare searches for Infogalactic + an article subject on Google and on DuckDuckGo.
At any rate, there's no need to have a lengthy argument about this. We'll find out soon whether it was a technical problem or not, based on whether fixing it turns out to be possible. Tetrapteryx (talk) 14:07, 22 September 2024 (UTC)
- I have created dozens of articles for dietary crackpots but some of those are still showing up on Google. For example David Perlmutter's article has been sitting comfortably on Google on the 2nd page for about 3 years whilst Eric Berg has been removed from Google. If there was an intentional action from Google it doesn't explain it for me why some biographies have been removed but others still remain. It's possible that it might only be the high-traffic articles removed but that doesn't fit all the deindexed articles. Having searched for 100s of articles it does look like about 90% have been removed from Google which is very high. I am inclined to believe this is a serious technical issue. In regard to when this first happened it appears to have been about 11 days ago. The question is what changed then and why did this suddenly happen. David Gerard is probably the only one who can solve this. Johns (talk) 15:00, 22 September 2024 (UTC)
- I believe Google is deindexing all mainspace articles. The reason some are not yet deindexed is because the automatic crawler has not yet crawled some pages. This reason I think this is some articles only vanished yesterday and 2 days ago, while many others a week ago. Diana Fleischman's article was 1st page on a Google search 2 days ago - but now its gone. You can see more articles are vanishing each day by searching site:rationalwiki.org.216.238.107.93 (talk) 15:21, 22 September 2024 (UTC)
- Actually pages are reindexed and deindexed every day. Not all of them, but some of them, as Google crawls a portion of pages every day. Some pages people have listed here as deindexed have appeared again. Yesterday Conservapedia was reindexed, today it is deindexed again. A logical fallacy page I used as an example of a deindexed page is currently reindexed. (It's likely some non-mainspace pages are also hit by the way, we just haven't checked as many.)
- I believe Google is deindexing all mainspace articles. The reason some are not yet deindexed is because the automatic crawler has not yet crawled some pages. This reason I think this is some articles only vanished yesterday and 2 days ago, while many others a week ago. Diana Fleischman's article was 1st page on a Google search 2 days ago - but now its gone. You can see more articles are vanishing each day by searching site:rationalwiki.org.216.238.107.93 (talk) 15:21, 22 September 2024 (UTC)
- Sometimes Google's bot succeeds in fetching pages, at other times it doesn't. When it doesn't, it's important what kind of error it gets. 500, 503, or 429 will make it slow down and try again. But 403 will make it deindex the page until and unless it gets a different response next time -- this is where all the "ouch" comes from. (No more news from David yet, more on that when it happens below.) --ApooftGnegiol (talk) 17:29, 22 September 2024 (UTC)
Seems purely technical, more investigation to be done[edit]
With help from David Gerard a first quick look at Google Search Console has been possible. Some URLs are indexed, some are not. The explanation on those not indexed seems to be this: "Page fetch: Failed: Blocked due to access forbidden (403)". This has the appearance of a technical mishap, not any kind of issue having to do with legal matters. We're looking more into it. --ApooftGnegiol (talk) 23:35, 21 September 2024 (UTC)
- Interesting I looked that line up and there are hardly any hits, this is the only webpage answer [1] "Most of the time a 403 status is due to a kind of firewall that blocks Googlebot from accessing your site / server. Check firewalls / security plugins / CDNs / other server settings". Possible solution reply "So the problem was that Mod security filter blocked different IP's including Google Bot. Solution was to whitelist the IP in htaccess file and now everything is working well!" Johns (talk) 01:53, 22 September 2024 (UTC)
- What's changed over this year I think is mainly various efforts and tweaks to block DDoS attacks, so a main guess is that some detail went wrong with that, which needs to be corrected. I talked to David earlier about bots, but he says Google's bot is well-behaved and not among the IP ranges he's targeted. But there's also a general rate limiter added earlier too, which could have something to do with it (my guess, we'll see what David has to say later). --ApooftGnegiol (talk) 02:26, 22 September 2024 (UTC)
what I've just done is disabled all the various bot blocks I put in recently so ApooftGnegiol can see what the Google console does. If the site melts, welll that'll be instructive too - David Gerard (talk) 18:35, 22 September 2024 (UTC)
I'm not so sure about that[edit]
It used to be that if you Googled "Natural History of Ashkenazi Intelligence" the Rational Wiki article was on the first page of results. Now, not only is it on no result page, it isn't even found via "Natural History of Ashkenazi Intelligence" AND Rational Wiki. Putting the article's URL also will not show the article, although it will find Rational Wiki talk pages about the article. What I did find when I Googled Natural History of Ashkenazi Intelligence, on the very last page of the search results, was this:
In response to a legal request submitted to Google, we have removed 1 result(s) from this page. If you wish, you may read more about the request at LumenDatabase.org.
Although there was no direct reference in the database to Rational Wiki. Pinkerite (talk) 20:40, 22 September 2024 (UTC)
- Lmao that's fucking weird. The sender was the American Chemical Society. The target was Sci Hub. Do we cite Sci Hub anywhere? Carthage (talk) 20:42, 22 September 2024 (UTC)
- We avoid pirate site links in article refs AFAIK (unless you count crank archive sites that gather conspiracy material etc.). Some editors probably use Sci-Hub but then cite the legal publisher, but that's a different matter and we couldn't police it even if we wanted to. --ApooftGnegiol (talk) 21:52, 22 September 2024 (UTC)
- This is part of a series of DMCA takedown requests that seem to be targeting piracy websites, which bizarrely have included requests for RW pages among the specified URLs. I had actually been monitoring this for a little while and as you can see if you go through each request, none of them were granted for RW URLs: https://transparencyreport.google.com/copyright/domains/rationalwiki.org
- So no, that's not it. Most likely the removed URL from this Google search Pinkerite did was a link to a site like Sci-Hub, not RW. Chillpilled (talk) 20:57, 22 September 2024 (UTC)
- I checked the mentioned RW article URL in Search Console, it's the same as the others with 403 error on fetch, that one was last crawled Sept 14. That pattern of the article not showing up no matter how you type the query is exactly what it means for any page to not be indexed. We'll see if it reappears later. --ApooftGnegiol (talk) 21:04, 22 September 2024 (UTC)
- Natural History of Ashkenazi Intelligence appeared for me when I Googled it. Chef Moosolini’s Ristorante ItalianoMake a Reservation 02:43, 23 September 2024 (UTC)
- I checked the mentioned RW article URL in Search Console, it's the same as the others with 403 error on fetch, that one was last crawled Sept 14. That pattern of the article not showing up no matter how you type the query is exactly what it means for any page to not be indexed. We'll see if it reappears later. --ApooftGnegiol (talk) 21:04, 22 September 2024 (UTC)
Google stats, one good thing to expect after this ordeal[edit]
In the past people have asked from time to time if we have stats for what people read on RW, and we haven't had it available. Now Google Search Console is building stats beginning quite recently, so we'll be able to share the "searched on Google" part of the picture in the future. For now, it seems one of our biggest hits is Project Blue Beam, on a conspiracy theory many search for, and that it's currently only second to the main page in how many visit after a search.
There may be huge gaps in stats given many pages are not yet reindexed, but we'll see later how it turns out. Also, I think it's viable to hand out access to check Search Console stats to more people on RW, it's possible to add any number of non-admin users who can then look at it. Something to think about and discuss later. --ApooftGnegiol (talk) 21:52, 22 September 2024 (UTC)
- By the way, we have 3x as many clicks from Google search as before we fixed this issue, but now that growth is plateaued, while indexed pages keep growing (but more slowly). --ApooftGnegiol (talk) 10:56, 1 October 2024 (UTC)
- Seems we set a new record for recent years as determined by Google measuring clicks from its search results. In Jan 2022, we got a "150K clicks from Google Search in the past 28 days". Now we got past a 180K marker. So despite having done better in the 2010s (probably before the reach of these measurements by Google), we're not doing too badly. --ApooftGnegiol (talk) 01:08, 21 October 2024 (UTC)
Some of the articles are back[edit]
Some of the deindexed articles are back e.g., Nigel Farage, Bryan Pesta and John Fuerst (mentioned above) are back with the latter two on top/1st page of Google. However, many others are still not indexed but it may now just be time it takes to crawler to update. I will check back in 48 hours and then a week. 161.35.171.161 (talk) 02:18, 23 September 2024 (UTC)
- Paul Saladino's article is now back on the first page of Google. This is a step in the right direction. I hope the others are also fixed. I will check back in a few days. This is very likely an issue resolved. Johns (talk) 11:19, 23 September 2024 (UTC)
- Many articles mentioned above still deindexed (Diana Fleischman, Emil Kirkegaard, Anatoly Karlin) so we will need to wait. The Bo Winegard though has just appeared back on Google. 161.35.138.212 (talk) 11:46, 23 September 2024 (UTC)
- The first two you mentioned are showing up on results to me now, but not Karlin's, and Emil's oddly doesn't have a preview of the page. Chillpilled (talk) 09:07, 24 September 2024 (UTC)
- Noah Carl's and Emily Willoughby's are still not showing either but Jonathan Anomaly is now back up. I guess more will be indexed over next few days or week.158.58.172.45 (talk) 19:15, 24 September 2024 (UTC)
- The first two you mentioned are showing up on results to me now, but not Karlin's, and Emil's oddly doesn't have a preview of the page. Chillpilled (talk) 09:07, 24 September 2024 (UTC)
- Many articles mentioned above still deindexed (Diana Fleischman, Emil Kirkegaard, Anatoly Karlin) so we will need to wait. The Bo Winegard though has just appeared back on Google. 161.35.138.212 (talk) 11:46, 23 September 2024 (UTC)
- At its worst (before David did the change to unbreak this) Google tracked only around 2000 URLs for RW in total. Now it's a little over 3x as many. Still plenty to go, but the trend is good. --ApooftGnegiol (talk) 11:02, 25 September 2024 (UTC)
- Is this just me or is Google actually acting quite strange in general lately? For topics I'd usually get about ten or whatever pages of results for, it's returning just one. Chillpilled (talk) 11:52, 25 September 2024 (UTC)
- I was wrong about the stats, recently some previously-blank sections in Google Search Console have been filled in with more after some days of data crunching in Google's system. The stats for HTTPS URLs is still below 8000 counted, but now there's separate indexing and indexing-problems stats too, with 73K pages indexed and 70K pages not indexed due to 403 error. Google's system will now go through those 70K and try to validate that this is fixed over time. (And yes, the non-indexed pages are not only mainspace pages but all over.)
- Is this just me or is Google actually acting quite strange in general lately? For topics I'd usually get about ten or whatever pages of results for, it's returning just one. Chillpilled (talk) 11:52, 25 September 2024 (UTC)
- Personally, I don't use Google search that much in recent years (mainly DuckDuckGo instead), so I don't have as clear an impression of how its results have generally changed. --ApooftGnegiol (talk) 14:35, 26 September 2024 (UTC)
- First off, 'Jonathan Anomaly' and 'Anatoly Karlin' is both currently #2 in Google UK [from my check as of thirty mins ago], so it's not that. [I'm in the UK for reference].
- Personally, I don't use Google search that much in recent years (mainly DuckDuckGo instead), so I don't have as clear an impression of how its results have generally changed. --ApooftGnegiol (talk) 14:35, 26 September 2024 (UTC)
- Part of me me suspects this 'hit-flop' is part that Google Search has degraded itself; that we're getting lower/missing rankings because it's increasingly becoming enshittified [like Apooft, I've shifted away from Google of late for searches not mainly related to social media content because the enshittification has gotten to the point Google has been serving up 'what I thought you said' results like it was a deaf grandma rather than y'know, what I actually asked].
- The issue is that [if the free SEO tools I played about with are right] it's not really Gerard's anti-DDoS hack/patch to blame, as the massive fall-off of hits was around '20-'21, when that hack was only put in late '23 if I remember right. I mean literally, a pre-Covid RW was 400-200k hits a month and a post-Covid one was 40k [if my half-drunk brain remembers right, we'd managed to clamber back to about 80k by late '23]. We'd fallen down to 40k again by summer '24 [which could be explained by the anti-DDoS action] but what about the rest? Out of peverse curiousity put in CP too and they'd suffered a hit too [less as a % of us because they're much smaller and thus I suspect more of their hits are direct links than engine results]. But I have an alternative theory, if you'd indulge a person who's majority of meagre technical knowledge is pretty obsolete...
- RW has become depreciated as a worthwhile source due to Google's LLMs.
- We've all heard about 'AI' giving shit advice like eating rocks and glue pizza. But where did it get this advice from? Sites it considered 'reliable', that's what. I suspect that it very might have decided to rank RW's reliability not on say, actually checking our mainspace articles for accuracy but instead going on opinions such as the Qurora result [2] or the places which call us as a 'distaff CP' [the balance fallacy in action!] without noticing that we've significantly evolved over time . I mean, it's nice to know that CP is not considered reliable for anything, but I wonder whether Google's LLMs have decided without actually checking us out whether said designation for us is in fact accurate.:::
- Which leads to the question; if my theory is at least partly correct, how can RW convince the Google LLM that we're not as shit a source as CP is? KarmaPolice (talk) 03:21, 8 October 2024 (UTC)
Well, why not look more at this broader topic now that the deindexing issue is fixed for now. To clarify, the long-term drop in popularity for RW and the short-lived deindexing issue are totally unrelated. You mentioned the long-term thing but others haven't discussed it much yet...
Google's LLM antics are newer than the great '20–'21 drop-off you mention, so those two are unrelated. The integration of Google's LLM into their search, rather than Google just scraping stuff to train LLMs with, is of this year. Or do you mean some additional newer drop-off?
Personally I can think of two separate reasons that I think are both important:
- Mobile browsing. 2/3 of browsing nowadays is such I think. RationalWiki hasn't been good for that. Google also lists how many pages it deems give good vs. bad browsing experience, and most of our pages are good for desktop but bad for mobile. This could be fixed on our end tweaking the wiki install/config.
- Topics and relevance. In earlier years RW was more focused on pseudoscience stuff, later focus drifted towards politically themed figures and controversy. I think the latter is of greater interest to many of our members, but of lesser interest generally to those who search Google. During the pandemic years much of our conspiracy and pseudoscience coverage was stuck in the 2010s, frankly. Most of our Google traffic is directed at a handful of pages on conspiracy and pseudoscience.
--ApooftGnegiol (talk) 03:53, 8 October 2024 (UTC)
- There has long been discussion in the technical support archives about upgrading our wiki software for better mobile support. We now have the funds to hire someone to make needed software upgrades including better mobile support. The Board can now act on this hopefully. Bongolian (talk) 04:01, 8 October 2024 (UTC)
- The board as such has been very quiet of late, I've poked them about having a full meeting, and there's also some agreement on the idea, and I've volunteered to give it a go myself too if given server access as it seems a simple extension to install. --ApooftGnegiol (talk) 15:00, 8 October 2024 (UTC)
- Maybe I'm wrong though, there's several ways to configure it and the easiest may perform too badly, while accompanying server software reconfiguration is needed to set it up more like Wikipedia does. I'll ask David a few things about this too. --ApooftGnegiol (talk) 15:18, 8 October 2024 (UTC)
- The board as such has been very quiet of late, I've poked them about having a full meeting, and there's also some agreement on the idea, and I've volunteered to give it a go myself too if given server access as it seems a simple extension to install. --ApooftGnegiol (talk) 15:00, 8 October 2024 (UTC)
- MinervaNeue is best for mobile -- it's what Wikipedia uses. Duncan from Teflpedia (talk) 14:16, 8 October 2024 (UTC)
- That skin is already installed since some time, but the extension that automatically switches to it for mobile browsing isn't. The point being the immediate "default experience" for guests to the website is what counts the most. --ApooftGnegiol (talk) 15:00, 8 October 2024 (UTC)
- All results are Google UK / Firefox [Private Mode] after flushing my history.
- That skin is already installed since some time, but the extension that automatically switches to it for mobile browsing isn't. The point being the immediate "default experience" for guests to the website is what counts the most. --ApooftGnegiol (talk) 15:00, 8 October 2024 (UTC)
- 'Not even wrong' #4, with snippet.
- 'Ethan Ralph' is #1, with snippet.
- 'Diana Fleischman' is #3, with snippet.
- 'QAnon' is below #50, but has snippet if you search directly.
- 'Paul Saladino' is #15, with snippet.
- 'Eric Berg' is #9, with snippet.
- 'Kent Hovind' is #3, with snippet.
- It might be that Google-crawlers haven't got to QAnon yet, or it might be that RW is ranked below the various media outlets which were #1-#50. At no point were RW 'beaten' by any site I'd regard as inferior, though Saladino/Berg's low numbering is partly due to multi pages from the subject themselves [which makes sense if a little annoying].
- Anyway, I was thinking - is there any way for us to get RW internal stats? Two I'm interested in is a) top mainspace hits from RW's search feature and b) top RW searches which didn't produce a mainspace. Because we've got a decent amount of pages which won't ever see the light of day via a generic engine search and the latter would give a very good 'most demanded' page list.
- Another thing is wondering whether RW is getting 'punished' by Google algorithms not just due to a lack of mobile mode, but the considerable amount of stubby/unoriginal mainspace pages and the anti-DDoS hack-patch. That in for the former, their presence is dragging down our global 'ham' rating and the latter is upping 'this is a horrible site' one [I've been having intermittent issues with RW pages of late - not just slow loads, but occasional no-load. KarmaPolice (talk) 16:01, 8 October 2024 (UTC)
- We don't have anything installed for providing such internal stats, which would first of all need to record and maintain that information.
- Intermittent issues preventing page loads are not due to anti-DDoS measures (which are currently disabled AFAIK), rather they are more likely due to the server being overloaded, which is what the anti-DDoS measures tried to prevent.
- I don't think having a bunch of poorer pages is that big of an issue, if it were it would cripple Wikipedia (which is way more lenient about stubs) which does great. --ApooftGnegiol (talk) 19:38, 19 October 2024 (UTC)
The captchas are getting unbearable[edit]
I understand that spambots can be a problem, however, the captchas on this wiki are so long and repetitive that on average I spend several minutes solving one, which makes it a nightmare to edit anything, or fix an error I made quickly, and even leave messages on people talk pages, I feel this can really drive new editors away due to how repetitive and irritating it is, which is a shame. While some captchas are fast to solve, it's really a 50/50 on whether I get a quick one or a long one where I spend more time solving the captcha than actually editing. I think we need a better captcha system for this, like Wikipedia's "type the scrambled letters" one, which doesn't take several minutes to solve, and from what I've seen combats bots well, or maybe even use an easier or better version of reCaptcha (if that's possible)? MethodsOfRationality (talk) 15:59, 8 October 2024 (UTC)
- Planned changes include changing that. No idea if we even need a captcha, I think bots can really go past all of them nowadays (but the crudest bots are still stopped) except the ones you craft where unique questions are used (until a spammer adds the solutions so it's automated). Spam IP blacklisting is something we can add drawing on a big database that I think does more.
- This is among stuff that's remained the same for a year. But maybe I'll get server access soon myself and then I can try changing it. The RMF board may also look into hiring someone if we can't more generally sort stuff out short term. --ApooftGnegiol (talk) 16:30, 8 October 2024 (UTC)
- @ApooftGnegiol Yeah I do believe AI can actually solve Captchas better than humans these days, also I'm not even sure if this wiki even has a bot problem. I like the idea of asking a unique question in the form of an image (e.g a photo asking for 8+8), not sure where I saw it though, but it was easy enough and not a hinderance. The MoR (talk) 13:22, 9 October 2024 (UTC)
- The wiki does have a bot problem, although it's not nearly as bad as it was when I first got started here. Chef Moosolini’s Ristorante ItalianoTry our pumpkin spice lasagna! 13:39, 9 October 2024 (UTC)
- @DuceMoosolini oh, but to be fair, most bots these days can get pastcCaptchas, due to the rise of AI, though I read somewhere text based captchas seem to do a better job at keeping them away The MoR (talk) 14:06, 9 October 2024 (UTC)
- It's more that even uncomplicated custom captchas work better than "good" off-the-shelf captchas because no one's written the code to target them. ikanreed 🐐Bleat at me 17:42, 17 October 2024 (UTC)
- What's really funny is that some of the most effective captcha-solvers are just companies paying people in the Global South to do nothing but solve captchas all day. Carthage (talk) 17:47, 17 October 2024 (UTC)
- @DuceMoosolini oh, but to be fair, most bots these days can get pastcCaptchas, due to the rise of AI, though I read somewhere text based captchas seem to do a better job at keeping them away The MoR (talk) 14:06, 9 October 2024 (UTC)
- The wiki does have a bot problem, although it's not nearly as bad as it was when I first got started here. Chef Moosolini’s Ristorante ItalianoTry our pumpkin spice lasagna! 13:39, 9 October 2024 (UTC)
- @ApooftGnegiol Yeah I do believe AI can actually solve Captchas better than humans these days, also I'm not even sure if this wiki even has a bot problem. I like the idea of asking a unique question in the form of an image (e.g a photo asking for 8+8), not sure where I saw it though, but it was easy enough and not a hinderance. The MoR (talk) 13:22, 9 October 2024 (UTC)
Name change request[edit]
- This discussion was moved to RationalWiki:Requests to change username. --ApooftGnegiol (talk) 17:34, 15 October 2024 (UTC)
Getting rid of RationalWiki:LiquidThreads?[edit]
RationalWiki:LiquidThreads, besides being years out of date, is currently being uninstall from WMF wikis (see T332022 and its subtasks). I am admittedly a very casual user of RW, but I do not see any current uses of LQT.
If it is not being used anywhere, uninstalling it in parallel with WMF wikis is probably the best time to do this. Installing DiscussionTools would also be nice :)
Best, HouseBlaster (talk) 00:29, 17 October 2024 (UTC)
- I think a couple of user talk pages use it, though the vast majority of people don't. I think just not using LiquidThreads like we all are currently doing should be enough unless the extension has security issues or causes server slowdown. Plutocow (talk) 04:38, 17 October 2024 (UTC)
- In the past a few used it, nowadays it doesn't seem to work properly if I remember correctly from some talk page I saw earlier (edit: could just be someone couldn't get it to work more recently, old pages using it display fine etc.). It sounds like a good idea to edit such old discussions so they end up on regular subpages (like WMF also does), then uninstall the extension later when dealing with updates. Less to worry about if the extension gets deprecated and unmaintained in the future, as otherwise it'll just be dumped whenever it breaks and we update.
- Problem is, it will be lots of clicking and editing to convert those pages, unless somehow it's done by a bot. There's plenty of pages even though few users. --ApooftGnegiol (talk) 16:59, 17 October 2024 (UTC)
- To be clear, the extension is currently unmaintained. The WMF is currently setting an undeployment sequence (LQT is incompatible with temporary accounts, which are tentatively scheduled to be rolled out to all WMF wikis by May 2025). HouseBlaster (talk) 22:02, 14 November 2024 (UTC)
Frivolous legal complaints this month to Google to deindex article, can it be identified who is sending these?[edit]
Hi,
Someone this month from the US has filed multiple frivolous defamation complaints to Google to deindex an article. Can we identify who is doing this?
Lumen have redacted their name.
Under the Defamation Act 2013 this attached site breaches a number of laws within this act. As detailed below
There must be a claim or assertion suggesting that someone has engaged in illegal, unlawful, or improper conduct, often made without concrete proof. There are many claims within this URL that claim [REDACTED] [REDACTED] has acted improperly i have attached in the following box specific text that infringe my rights.
As part of the Defamation Act 2013 any claim must be directed clearly towards a specific, identifiable individual or entity .The claims and assertions via the attached URL are directed towards [REDACTED] [REDACTED] a number of times with mentioned and tags to her name.
As part of the Defamation Act 2013 any claim or assertions must lead others to view the party negatively, thereby harming their standing in the community. This content via the attached URL labels me [REDACTED] [REDACTED] as a " NEO NAZI" and " a supporter of eugenics" which leads 3rd parties to view me in a negative and reputation damaging way. The claims are false and there is no evidence attached for 3rd party to view such assertions or claims to therefore make their own decision or judgement so it directs 3rd parties to view me negatively immediately without providing a full scenario.
There must be verifiable evidence of harm or loss that has resulted from the defamation, which can include tangible losses like financial harm or intangible ones like emotional distress. There is verifiable evidence of loss of job opportunities and funding for my studies and academia, i have been told a number of times that whilst this URL sits on Google they will not associate with me. This under the Defamation Act 2013 infringes my rights.
This URL also associates me with organisations , conferences and other academics that i have no association with which defames my character, there are some controversial conferences and academics and there is no proof at all of me attending or associating with any of these academics named.
The offending URL attached is a bias , reputation damaging site that specifically targets and abuses academics , they right unsourced comments and label myself and others in a defaming manner and they do no respond to communication to edit there mistakes and false content. It is a community site meaning hundreds of users have access to edit the site therefore i cannot pinpoint directly who to direct a defamation case to therefore the hosting site must be held responsible for not controlling users who write defaming material.
- https://lumendatabase.org/notices/45115564 (October 1 2024)
The more recent:
The attached URL shares false, misleading and unsourced claims about [REDACTED] [REDACTED] and also refers to a child birth name which was altered for legal reasons and should not be shared in the public domain as it imposes risks and threats to me and infringes my human rights. The site in questions is a public editorial site open to any member of the public to edit anonymously and therefore cannot be held accountable for the content on their site and as a result do not reply or deal with any enquiries to change or alter illegal or misleading content.
Within the attached URL there are a number of unsourced allegations and misleading claims which results in reputation damage and loss of opportunity and discredit of academic work. I have evidence to support this claim . Many statements and claims are cited with a broken URL to a fake page so they cannot be deemed reliable and this site simply advertises all such claims about me illegally and does not control or moderate the pages within .
Unproven and unsourced claims about me , details of where i work which are also false and again mislead 3rd parties of interest.
- https://lumendatabase.org/notices/45510487 (October 14 2024)
Any ideas? These complaints are very dishonest and are not even legally consistent since they were filed from the US but the person is mistakenly talking about the UK Defamation Act 2013. This does not apply to defamation law in the US (you can only block content/deindex pages on Google from the country where you live and file the complaint from). 45.76.124.89 (talk) 15:17, 19 October 2024 (UTC)
- I thought at first this could be Jonathan Anomaly who filed a frivolous complaint in 2021 (see his talk page or here). Anomaly's birth name is Jonathan Beres and the complaint above mentions "child birth name" and eugenics. However, if you compare the complaint Anomaly filed in 2021 to the above, the writing style is very different. 45.76.124.89 (talk) 15:22, 19 October 2024 (UTC)
- Those filings are mostly illiterate with plenty of amateur mistakes and lies. The filings are likely to be done by someone associated with the Human Diversity Foundation. I am currently writing a RW article for the Human Diversity Foundation, it will be out in the next 48 hours. The person filing that is making things up. None of our RW articles refer to any specific person within this group as a "neo-Nazi". All the articles are well sourced to multiple media exposes and newspaper sources. The lumendatabase does not work for people in the US. Whoever is filing these is clueless and is doing so in bad-faith. Psychologist Guy (talk) 15:35, 19 October 2024 (UTC)
- On further investigation the complaint refers to a "her". I cannot find any RW articles referring to any female as a "neo-nazi" and "supporter of eugenics". The complaints are likely hoaxing/trolling attempts to promote misinformation about RationalWiki. Psychologist Guy (talk) 15:50, 19 October 2024 (UTC)
- The "her" may have been a typing error. The only woman I can think of is Carolyn Emerick but she publishes under her real name so it would not make sense she would complain of a "child birth name" on the page. She's also not an academic but a crackpot. Who knows. I think it remains possible this was filed by someone linked to Anomaly, however, I am far from convinced. The only other person I can think of is Angelo John Gage aka Lucas Gage who changed his name. Both "neo-Nazi" and "supporter of eugenics" are found on his article, however, he is not remotely an academic. Anti-eugenicist (talk) 15:56, 19 October 2024 (UTC)
- Could it be Tara McCarthy? Her birth name is Tarasita Hanrahan (as mentioned on page). However, she is not an academic but another crackpot. Anti-eugenicist (talk) 18:07, 19 October 2024 (UTC)
- Is there some method that we [as RW] could respond to Google regarding this complaint? I suspect if someone contacts Lumen 'as RW' and asks to know who, we might find out [though there might be conditions attached] KarmaPolice (talk) 18:51, 20 October 2024 (UTC)
- The "her" may have been a typing error. The only woman I can think of is Carolyn Emerick but she publishes under her real name so it would not make sense she would complain of a "child birth name" on the page. She's also not an academic but a crackpot. Who knows. I think it remains possible this was filed by someone linked to Anomaly, however, I am far from convinced. The only other person I can think of is Angelo John Gage aka Lucas Gage who changed his name. Both "neo-Nazi" and "supporter of eugenics" are found on his article, however, he is not remotely an academic. Anti-eugenicist (talk) 15:56, 19 October 2024 (UTC)
- On further investigation the complaint refers to a "her". I cannot find any RW articles referring to any female as a "neo-nazi" and "supporter of eugenics". The complaints are likely hoaxing/trolling attempts to promote misinformation about RationalWiki. Psychologist Guy (talk) 15:50, 19 October 2024 (UTC)
- Those filings are mostly illiterate with plenty of amateur mistakes and lies. The filings are likely to be done by someone associated with the Human Diversity Foundation. I am currently writing a RW article for the Human Diversity Foundation, it will be out in the next 48 hours. The person filing that is making things up. None of our RW articles refer to any specific person within this group as a "neo-Nazi". All the articles are well sourced to multiple media exposes and newspaper sources. The lumendatabase does not work for people in the US. Whoever is filing these is clueless and is doing so in bad-faith. Psychologist Guy (talk) 15:35, 19 October 2024 (UTC)
Periodic sluggishness[edit]
I've been seeing periodic sluggishness in response times in the past few hours. Bongolian (talk) 06:42, 4 November 2024 (UTC)