Planet Wikimedia

Technical news

• [Michael Jackson Kills WP] - Michael Jackson, the "King of Pop", died this week and nearly took Wikipedia down with him (as well as many other websites).  Wikimedia sites were unresponsive for a whole due to the large number of page hits.  The Michael Jackson article got nearly 6 *million* hits on June 26, more than the Main Page.
  

• [Usability] - the Usability team has released its first set of usability improvement (the "acai" release) for testing.  As of press time, the new "vector" skin has been made available on all wikis for users who select it from their preferences, please help test the new features!
• http://techblog.wikimedia.org/2009/06/first-usability-release-is-coming-up-soon/ -- blog post
• http://usability.wikimedia.org/wiki/Releases#Acai -- "acai" release description
• [Usability BIS] - one aspect is already live by default; the new design of the search results. Just use the internal search and see.
• http://usability.wikimedia.org/wiki/Designs#Search_Results
  

Request for help

• [Fundraising 2009] - the Wikimedia Foundation is preparing for its next fundraiser and would like feedback from the community on some of the pages it has developed so far.  In addition to the creation of a new survey that they would like comments on, the Foundation is also trying to enhance the visibility of the donate button. The new buttons will be placed on the skin and will be viewable on every page. There are also several design proposals that need community input!
• http://blog.wikimedia.org/2009/06/25/would-you-press-this-button/ -- blog post about it
• http://meta.wikimedia.org/wiki/Fundraising_2009/Donation_buttons_upgrade -- buttons
• http://meta.wikimedia.org/wiki/Fundraising_2009/Survey -- survey
  

Foundation

• [Licensing] - The "licensing update" changes have now been rolled out to all languages and projects, with the new license being featured in the footer and new interface items to translate for your wiki.
• http://blog.wikimedia.org/2009/06/30/licensing-update-rolled-out-in-all-wikimedia-wikis/ -- blog post
• http://lists.wikimedia.org/pipermail/translators-l/2009-June/000959.html -- translation steps
• http://tinyurl.com/LU-interface-trans -- betawiki trans interface
• [Board election] - Once again an election for the board of trustees of the Wikimedia Foundation is coming up. It is for 3 seats this time. But before there can be elections there must be someone to vote for. Submissions of candidacy are open between 6th and 20th July.
• http://meta.wikimedia.org/wiki/Board_elections/2009
• [New grant: Commons] - the Wikimedia Foundation was just awarded a US$300,000 grant that will research problems with and design new tools/fix current tools for uploading files to the global Wikimedia image repository, the Wikimedia Commons.
• http://blog.wikimedia.org/2009/07/02/ford-foundation-awards-300k-grant-for-wikimedia-commons/ -- blog post
• http://wikimediafoundation.org/wiki/Press_releases/Wikimedia_Ford_Foundation_Grant_July_2009 -- press release
• http://wikimediafoundation.org/wiki/File:WMF_Ford_Multimedia_Participation_Project.pdf -- full grant proposal
• [Job: "Bookshelf"] - as a further development from the "Scribus operator" volunteer position noted in Wikizine a few months ago, a new job opening has been posted for project manager of the "Bookshelf" project.  The "Bookshelf" project strives to develop a slate of basic educational materials (print, online and video) to attract new authors and editors to Wikipedia.
• http://wikimediafoundation.org/wiki/Job_openings/Project_Manager_Bookshelf -- job opening
• http://meta.wikimedia.org/wiki/Public_outreach/Get_involved -- volunteer position, scribus operator
  

Community

• [David Rohde] - it turns out that Wikipedia, along with tons of other media outlets, kept the kidnapping of journalist David Rohde out of its article on him.  This has caused a few Wikipedians and Wikipedia critics to note this and either scream censorship or just point out the fact that it was unsourced information that could help save someone's life by erring on the side of caution.
• http://www.nytimes.com/2009/06/29/technology/internet/29wiki.html?ref=media -- nytimes article
• http://features.csmonitor.com/innovation/2009/06/29/was-wikipedia-correct-to-censor-news-of-david-rohdes-capture/ -- blog post about censorship
• http://yro.slashdot.org/story/09/06/29/2120257/Wikipedia-Censored-To-Protect-Captive-Reporter -- slashdot discussion
• [A reliable article?] - A new tool tries to help make scholars feel better about using Wikipedia by telling how reliable an article seems to be by using "wikibu-points".  These points are based on statistical criteria and represent only possibilities, the creators hope that warnings will help people think to look at the discussion pages or article history.  Wikibu is only available on the German Wikipedia at this time.
• http://www.wikibu.ch/ -- site/tool
• http://lists.wikimedia.org/pipermail/wiki-research-l/2009-July/000833.html -- mailing list post
• [EN Wikibooks] - is now using Meta's unified "user language" template. It will enable greater collaboration on the textbooks that teach foreign languages.
• [enwp AFD] - An interesting AFD (deletion discussion) on the English Wikipedia was recently closed -- it resulted in the deletion of 4077 articles, probably the most ever from a single AFD on enwp!
• http://en.wikipedia.org/wiki/Wikipedia:Articles_for_deletion/Anybot's_algae_articles -- deletion discussion
  

Media

• [No tourists] - an article by a New Zealand newspaper tells that "references to gang violence and crime on Palmerston North's Wikipedia page have seen overseas investors and professionals shy away from the city"!
• http://www.stuff.co.nz/technology/2516472/Wikipedia-entries-slag-off-Palmerston-North
  

Stats

• [bn.wp] - The Bengali Wikipedia (bn) has reached 20,000 articles.
• http://bn.wikipedia.org/wiki/আন্তর্জাতিক_প্রকৃতি_ও_প্রাকৃতিক_সম্পদ_সংরক্ষণ_সংঘ -- 20,000th article
  

Other news

• [Who owns transit schedules?] - Only marginally related but interesting nevertheless, the subject of a recent battle has been pretty simple: who owns the copyright to bus arrival times?  Public transportation agencies are trying to assert that they own the times and stop people from making iPhone apps (and similar items) off of the content, so that they can do it themselves and charge people.  Not very FOSS of them!
• http://www.techdirt.com/articles/20090628/1419595382.shtml
• [Wikizine@Foundation-l] - Wikizine will now be featured on foundation-l too!  Thanks to Milos for proposing it and the list members for agreeing.
• http://news.gmane.org/gmane.org.wikimedia.foundation
• [Firefox 3.5] - ... is finally out. This main stream browser supports natively Ogg Theora and Ogg Vorbis, the media file formats our projects are using.
• http://www.mozilla.com/
  

Quote

"Programming languages are like cats. It is easier to get a new cat than to get an old cat fixed." -- Douglas Crockford

Editor(s): Casey, Walter , Corrector(s): Alex , Thanks to: Rand, Steve Bennett, Brion, Domas, Nando, Erik, Belayet, Naoko, Chad, Kul, Jay, Signpost, Adrignola, rainman-sr , Contact: reply or http://report.wikizine.org , Website: http://www.wikizine.org , Wikizine.org makes no guarantee of accuracy, validity and especially but not limited to, correct grammar and spelling. Satisfaction is not guaranteed. Wikizine.org is published by [[meta:user:Walter]]. Wikizine is a irregular publication as long as there is noteworthy news (and time) Content is available under the GNU Free Documentation License http://www.gnu.org/copyleft/fdl.html and also the Creative Commons Attribution 2.5 Generic License http://creativecommons.org/licenses/by/2.5/  

This seems to be a power outage at our European proxy caching cluster; we’ll see if we can give more details later.

European traffic has been rerouted to our US servers, but the extra load may cause the sites to be a little sluggish for now. (If your DNS is still seeing the old entries, you can manually configure your browser to use the US proxy: rr.pmtpa.wikimedia.org port 80. You should only do this temporarily, as you won’t be able to access anything *but* Wikipedia and our sister projects. :)

Update 21:13 UTC:

European servers are coming back online, we should have this cleaned up pretty soon.

Update 21:26 UTC:

We’re starting to switch traffic back to Europe. Should be better in a few minutes… In the meantime, amuse yourself reading the Twitter panic. :)

Update 21:40 UTC:

You can also use the SSL interface to Wikipedia, which doesn’t have the proxy overload.

Hi all,

Some of you might have already seen my blog posts about LiquidThreads, Wikimedia’s in-development discussion system.

For those who haven’t, this is a quick primer on what LiquidThreads is, and what it’s going to do for Wikimedia’s communities.

Currently, Wikimedia’s discussion system sucks. Here’s why:

• It’s not easily usable by the average user. It isn’t obvious how to leave a comment on a talk page, or how to reply to a comment. The indenting we use now is ad-hoc and unsustainable for long discussions.
• Signatures are done manually and we have to jump on poor unsuspecting newbies who don’t know this (or write bots…)
• Archiving is done unevenly by bots, which are maintained by users and therefore of very uneven quality. Archives are something of a black hole — they aren’t searchable, easily maintainable or easily accessible. You can’t resurrect an archived discussion easily, nor can you view its history.
• It’s stored as plain wikitext, which is opaque to any sort of automated process.
• You can’t move a thread to a different discussion page and preserve its history.
• There’s no encouragement, mechanism or incentive for quoted, point by point inline replies like we’re all used to with e-mail.

Enter LiquidThreads. LiquidThreads is a system that makes MediaWiki’s discussion system behave like a forum or comments thread, while still maintaining the unique refinements that make wikis work. It was originally designed by a Google Summer of Code student, David McCabe, and I’ve been making incremental improvements to make it work for Wikimedia.

So, what’s changed?

• Comments are separated from each other in the wikitext, so there are no more edit conflicts in discussions, and the usability is vastly improved.
• Instead of indenting, each comment is in its own box, along with its replies. It makes it much easier to follow each post and its replies, and it’s much nicer on the horizontal whitespace. Hopefully, it will be the death of the ‘arbitrary section break’!
• Each post has its own history page, making it easy to see what’s going on with individual threads without trying to navigate the history of a whole page.
• It’s easy to move threads between pages, preserving the page history.
• Discussions  are never ‘archived’. Instead, older discussions fall to the bottom of the page, and eventually they drop off entirely, to hit a new page. If you missed the chance to have your say, just reply to a discussion and it’ll be bumped right up to the top of the page again!
• Discussions with recent changes are at the top of the page. Discussions that have fallen dormant fall to the bottom. It’s easy to find out what’s happening!
• You can watch individual threads of a discussion, and even get an email when they’re replied to.
• It’s easy to link to a discussion, and the links are permanent unless the discussion is deleted. There’s no need to point to an archive or to an old revision ID.

If you’re interested, I’ve put together a test setup for you to play with it.

As always, questions, comments and suggestions are more than welcome, in the comments or elsewhere.

The first usability release, Acai, hit Wikipedia and sister projects this afternoon. The new skin, Vector, and the enhanced toolbar can be turned on from the user preference under “Appearance” and “Editing”. Search result page now has a new layout with less daunting information. Vector is only available for left-to-right languages at a moment due to IE6 incompatibility. However, the enhanced toolbar can be selected from all languages and the new search result page is enabled globally. We could not roll out two features we had planned. First, warning messages for unsaved changes when a user switches away from the edit tab did not work properly thus they are disabled. So please be careful when you switch away from the edit tab. Secondly importing language specific configuration for special characters were not graceful, so we disabled special character function from the toolbar. We are working on the fixes and plan to roll them out as soon as we have stable solutions. The usability project wiki has Vector and the new toolbar as a default, so if you prefer to check them out without changing your preferences it is a good place to visit first. Let us know what you think. We would love to hear from you.

Best,

Naoko

I’m very happy to announce that the Ford Foundation has awarded a USD 300,000 grant to the Wikimedia Foundation to improve our interfaces and workflows for multimedia uploading. See the press release and the grant proposal as submitted (PDF).

This should give you a good idea about what we can do within the scope of this project. Wikimedia Commons , the multimedia repository shared by Wikipedia and all other projects operated by the Wikimedia Foundation, has been a wonderful success story, having grown to more than 4.5 million educational, freely usable media files since its inception in 2004. But the combination of the complexity of free content licensing and the integration of Commons into the experience of contributing to a project like Wikipedia or Wikibooks can make for a very daunting experience for new contributors.

We want to begin to change that, and make sure that everyone who has useful educational media to share can do so easily. As part of our partnership with Kaltura, Michael Dale has already done some great work on external repository searches and transfers, and on integration of uploading into the editing interface, so we’re hoping to build on top of this to really get the workflow for licensing/upload/review/embedding of media files nailed.

We’ve also been having initial discussions with some of the Wikimedia chapters about possible models for working together on the execution of this project. For example, we want to make sure that we can facilitate fruitful face-to-face meetings with Commons practitioners, and there is plenty of technical work to be done that can be decentralized and shared. Exciting projects like Wikimedia Germany’s investment in multilingual search (German link; see Google Translation) are already underway, so hopefully over the next year, we’ll see lots of useful activity culminating in genuine improvements for Commons and beyond.

Big thanks to Sara Crouse and Naoko Komura for their work on this grant proposal, and of course we’re enormously grateful to the Ford Foundation for funding it. Wikimedia Commons deserves to grow to many more millions of free educational media files, and hopefully this strategic investment will help us to get there.

Erik Moeller
Deputy Director, Wikimedia Foundation

There has been a lot of talk in the last year about the usability of the MediaWiki software. Today, the new functionality created by the Usability Initiative has been rolled out. Have a look at today's English Wikipedia main page..


It not only looks good, it also makes it easier to understand what you can do. Given the large percentage of people who just do not appreciate that they can edit a page, it is a big improvement.

When you want to be part of the future of Wikipedia today, you have to change to the "vector" skin. Another neat feature is the "enhanced editing toolbar" that you can enable on the "editing" tab of the user preferences.


At this stage the usability improvements are available for all the "left to right" languages. Because of incompatibilities in IE-6, some more work is needed before it will become available for languages like Arabic and Hebrewl.

All in all, it is a happy day because you can see the shape of things to come for all of us and you can experience it now. When you find any issues, this is where they are happy to leaarn about it.
Thanks,
       GerardM

With six projects in over 250 languages, multilingual communication and content translation are big priorities for us. That’s one reason I was excited to go to the Open Translation Tools 2009 conference and be in the same room with 80 other translators, content providers and developers all working in the open translation space. Another reason is that the conference was held in Amsterdam in the old city center, in a beautiful venue right by one of the canals.

We have some amazing opportunities to collaborate with folks on other projects, from translation memory based systems like that in use by the World Wide Lexicon to source code string repository interfaces like Transifex. As one person put it, the perfect testbed for crowd-sourced translation is Wikipedia; if we can’t make it work there, where can it work? I also had a chance to talk with Gerard Meijssen and Siebrand Mazeland about new ways to facilitate tighter integration with translatewiki.net and to encourage more projects to make use of the translatewiki facilities. It should be a really productive year.

Folks told me to go visit the Van Gogh Museum, so I was dismayed to find that they don’t allow photography. However, the Wiki Loves Art NL project, organized by the NL Wikimedia chapter, had reached an agreement with the museum to allow two small groups in for photographs, during the week I happened to be there! So, come Tuesday morning, I was one of 20 lucky Wikimedia community members and photojournalists to be given private access to the Van Gogh collection. Some photos from the group are already available on the flickr group from which they will be uploaded to the Commons.

Right after the conference I went to the first two days of the OTT book sprint, which had as its goal the production of a comprehensive manual for beginner volunteer translators of open content with open tools. Once again we were in an awesome venue (see the picture; we were in one of the turrets!) and under the expert guidance of Adam Hyde we got a huge amount of content generated in just a few days.

On the last day I skipped town to go visit a colleague on one of the Wikimedia projects; we’ve worked closely together for over two years and had never met face to face. Perhaps that was the most important part of the whole trip: bringing our virtual community into the real world one person at a time.

WIKICITIES, Helmand, Monday (NNN) — The kidnapping of Pulitzer Prize-winning New York Times journalist David Rohde in Afghanistan was suppressed not only by almost all press syndicates but also by Wikipedia, on the direct command-and-control orders of Jimbo Wales, who is personally responsible for every word in the popular web-based encyclopedia.

Conservative commentators were appalled at the suppression. “Would they have protected HITLER like this?” thundered Michelle Malkin. Wales pointed out that the encyclopedia’s biography of Hitler had already been appropriately edited and cited per the Biographies of Living Persons policy:

Adolf Hitler is the Chancellor of Germany^{[citation needed]}. He is noted^{[citation needed]} for his work on the moral fibre of German society^{[citation needed]} and stimulating the economy^{[citation needed]}, notably through the Autobahn construction programme^{[citation needed]}. Some^[who?] have criticized aspects of his policies^{[citation needed]}.

(Read more …)

Some brief traffic statistics on the Arbitration Committee's mailing list:

• a total of 14692 messages were received by the list from January through June this year
  
• an average of 81 messages were received each day
  
• this is more than foundation-l (4473), wikien-l (4015) and wikitech-l (2924) combined over the same period, with change left over
  

Conclude from this what you will.

We had 52 minutes of downtime on the English-language Wikipedia site today; only en.wikipedia.org was affected. Our master database server was thrown into a funky state in which hundreds of access threads were stuck in the “statistics” state — which seems to be MySQL’s way of saying “I’ve fallen and I can’t get up”.

It’s unclear exactly what set it off, but basically nothing works until you restart MySQL. After switching the site to an alternate master database, all has been well.

At 52 minutes from start of event, this took us a bit longer than I’d like to resolve — we had to percolate through a couple levels of alert calls before we finished diagnosing it and getting the DB switch pushed through. (Sorry to wake you up early Tim!)

A similar event in future should be fixable within a few minutes, thanks to Tim’s work on making the master-switch system more foolproof. We’re fixing up our internal documentation so all our site ops will now know  how to run the database master switch script next time!

– brion

Well, my latest foray into the featured article candidates process was a surprisingly smooth one, and my article is now featured. It was a long slog, but well worth it. When I started out, it looked like this. A mess, basically. Just over five months of hard work later, it is now a featured article. I learnt a lot during the process, and I'd like to share what I learnt with everyone.

1. Write about what you know, or what interests you greatly.
   In my case I wrote about where I live. I know the area extremely well, and while some might argue this may affect WP:NOR issues, I did not feel it did for me. Additionally, the books I used are not published in great numbers - this is generally the case for local history books. They are, however, available in the local library. Every single book used was from the library, and were invaluable in finding facts to write about. A final point is that I enjoy finding out about my local area. I have been interested in history particular, since a young age, so it was enjoyable to research and write about.
2. Take your time with it.
   This is a good lesson I learned. I wanted to take it to FAC earlier than I did, because I thought it was ready. However, it was clearly not, as I took it to peer review instead and got invaluable feedback. There is no need to rush it; there's no time limit, it'll still be there tomorrow.
3. Use free images.
   It is a free encyclopedia, right? Unfree images don't help the FAC, and take up a lot of time from what I have seen. On my FA, all the images are free ones.
4. Make use of other people.
   I was lucky enough to be working on an article that is part of a very active Wikiproject, with some of Wikipedia's best editors on it. I used them - frequently. There are some sections on the article that I did not even have to author, as someone else did them for me. Working on a FA does not need to be a solo process.
5. Get it to GA first.
   Some may disagree with this, but I think it is a good a way as getting feedback as any. And if it fails you know you have a long way to go. But you still have feedback to work on, which is good.

A final point: make sure you enjoy what you're doing, above anything else. Stuff the MOS, 1a criterion and other minor things while you're writing up the content. Deal with the nitty-gritty bits once you've put what needs saying. Good luck, should you try a FA yourself!

There's been a lot of coverage of the New York Times story "Keeping News of Kidnapping Off Wikipedia." It's prompted discussion about balancing issues of free speech, safety, and responsibility at the Times and Wikipedia. Within Wikipedia, the discussion has only just begun, but has started off quite constructively as seen in Wikipedian Apoc2400's proposed policy: in the short term, Wikipedia should refrain from spreading information if that information is not widely and reliably sourced, of little public interest, and is "likely to have very severe direct negative consequences."

After spending about 6 months in alpha-beta-development-maybe-kind-live mode, we have recently moved Wikipedia Mobile over to a new fast and sexy server. With this new server, we’ve reached the point in development where we can call this baby “launched”!

When I was brought on board at Wikimedia, I was tasked with endowing Wikimedia with a compelling mobile offering. From the beginning, we knew we were going to focus on “fully featured” smart phones. These phones are taking more and more of the market and we believe they will have an easy majority-share in a couple years. The goal is to build for the future.

At the moment, the Mobile site supports iPhone, Kindle, Android, and Palm Pre. And we fully support both English and German. There are other working languages, but they haven’t been fully translated yet. Our goal is to grow slowly and do it really well. We are starting out simple with limited support in order to test the usability and the platform’s stability. So far, things are looking good.

During the beta test period, we’ve served around 10,000,000 pages. You can view the hourly stats here (updated every hour on the hour). And with this new test server, we should be able to do more.

Based off of requests from Google and the Palm Pre folks… and with what just makes sense. We are doing default mobile redirects. That is, if you open a wikipedia link on a supported mobile device, then you get redirected automatically to the mobile gateway. If you click the “View this page on main Wikipedia” then we disable that redirect with a cookie. This way, the 99% of people using mobile devices to read Wikipedia on-the-go have a seemless experience. And, the 1% who like to edit on their mobile device can use their browser to view the main site and do all the fancy things that they like doing. We suspect an initial outcry from the editors that use their mobile devices, but hope that will calm down. We’ve had very good feedback from the 99% and so we can’t forget those folks. If anyone has any suggestions on how to make this easier for the 1% who are editing while mobile, we’d love to hear from you.

If you want live updates about the Mobile site then you can follow WikimediaMobile on Twitter. Also, if you know any Ruby, you can grab the source code via git from Github and helpout! Feel free to contact me via email with any questions.

Also, special thanks to Nic Williams and Ryan Bigg from Mocra for help with the Ruby 1.9 transition and thanks to Yahuda Katz for help with the XML parsing layer and for all his work on the Merb framework.

Congralutations are in order for our friends and comrades-in-arms at Mozilla: they’ve released version 3.5 of their open-source Firefox browser today.

Aside from major improvements to speed and memory usage, one of the updates that has got us most excited at Wikimedia is the support for HTML 5’s native <video> and <audio> elements.

What does this mean? Well in short, it means that Firefox 3.5 is the best browser to run video and audio clips from Wikimedia Commons on!

A few months more down the line, we’ll start being able to integrate support for our inline video sequencer, which’ll make it easy to extract snippets of a longer video and combine them — entirely using open-source, non-patent-encumbered web standards. This makes heavy use of the new HTML 5 multimedia support; while at first editing will be limited to Firefox 3.5 users, other browsers are continuing to improve and adopt the same support.

As many folks have noted, our current templating system works ok for simple things, but doesn’t scale well — even moderately complex conditionals or text-munging will quickly turn your template source into what appears to be line noise…

<includeonly><span style="white-space: nowrap;">{{#if:{{{3|}}}|
{{coord|{{{1|0}}}|{{{2|0}}}|{{{3|0}}}|{{{4|N}}}|{{{5|0}}}|{{{6|0}}}|{{{7|0}}}|{{{8|E}}}|{{{9|type:other}}}|format={{{format|dms}}}|display={{#if:{{{title|}}}|inline,title|inline}} }}| {{#if:{{{2|}}}|
{{coord|{{{1|0}}}|{{{2|0}}}|{{{4|N}}}|{{{5|0}}}|{{{6|0}}}|{{{8|E}}}|{{{9|type:other}}}|format={{{format|dms}}}|display={{#if:{{{title|}}}|inline,title|inline}}}}| {{#if:{{{4|}}}|
{{coord|{{{1|0}}}|{{{4|N}}}|{{{5|0}}}|{{{8|E}}}|{{{9|type:other}}}|format={{{format|dec}}}|display={{#if:{{{title|}}}|inline,title|inline}}}}| {{#if:{{{1|}}}|
{{coord|{{{1|0}}}|{{{5|0}}}|{{{9|type:other}}}|format={{{format|dec}}}|display={{#if:{{{title|}}}|inline,title|inline}}}}}}}}}}}}</span></includeonly><noinclude>
{{pp-template|small=yes}}
{{documentation}}
</noinclude>

And we all thought Perl was bad!  ;)

Lua

There’s been talk of Lua as an embedded templating language for a while, and there’s even an extension implementation.

One advantage of Lua over other languages is that its implementation is optimized for use as an embedded language, and it looks kind of pretty.

An inherent disadvantage is that it’s a fairly rarely-used language, so still requires special learning on potential template programmers’ part.

An implementation disadvantage is that it currently is dependent on an external Lua binary installation — something that probably won’t be present on third-party installs, meaning Lua templates couldn’t be easily copied to non-Wikimedia wikis.

There are perhaps three primary alternative contenders that don’t involve making up our own scripting language (something I’d dearly like to avoid):

PHP

• Advantage: Lots of webbish people have some experience with PHP or can easily find references.
• Advantage: we’re pretty much guaranteed to have a PHP interpreter available.  :)
• Disadvantage: PHP is difficult to lock down for secure execution.

JavaScript

• Advantage: Even more folks have been exposed to JavaScript programming, including Wikipedia power-users.
• Disadvantage: Server-side interpreter not guaranteed to be present. Like Lua, would either restrict our portability or would require an interpreter reimplementation. :P

Python

• Advantage: A Python interpreter will be present on most web servers, though not necessarily all. (Windows-based servers especially.)
• Wash: Python is probably better known than Lua, but not as well as PHP or JS.
• Disadvantage: Like PHP, Python is difficult to lock down securely.

Any thoughts? Does anybody happen to have a PHP implementation of a Lua or JavaScript interpreter?  ;)

– brion

Update:

Hampton reminds me that Ruby has some sandboxing features and may also be a contender.

This is our second on-location episode of Wikivoices, recorded "live" at Drexel University in Philadelphia.

At the recent meetup, a couple Wikipedians talked with a Swarthmore University Free Culture activist and a Philadelphia teacher about ideas for a Wiki Takes Philadelphia outreach project.


Hopefully in future, we can organize more Wikivoices episodes to be recorded live at meetups around the globe.

If folks have thoughts on the discussion and outreach ideas raised in this episode, or just the format of this type of episode, please add them to the comments.

And thanks to Marc for the much improved the audio quality and editing, much better than mine last time!

About two years ago, I said "Commons may be the next coolest project, as soon as developers find the time to improve its usability to make it more user-friendly". Wikimedia Commons hasn't evolved much in terms of usability since then. MIT's Technology Review recently published an article about improvements to come regarding the management of video content on Wikipedia and Wikimedia websites. I heard a lot of people say: "Good, but what about pictures?" Some technical improvements described by the Technology Review will be useful for both images and videos, such as the media and upload wizard currently developed by Michael Dale. However, Wikimedia Commons still needs many little (or big) features that would dramatically improve its user-friendliness.

Since my last post about LiquidThreads, I’ve given it a major visual makeover, making it much prettier, and more in line with what users are used to with comment threads, forums, and other similar software.

Here’s an overview of the new interface

The new interface relies on paging, rather than archiving. Instead of artificially removing a discussion from a talk page when it’s done, the discussion page is paged. Only ten discussions will be shown at a time, and anything beyond that can be accessed simply by hitting ‘Next’. Old discussions will naturally fall ‘off the bottom’ of the discussion page, as they do in forums, comment threads, and most other discussion systems in the world.

I’ve also redesigned the thread display. Instead of having a footer with the pertinent information, and using indentation to show threading, I’ve put the information at the top of an enclosing box, which includes all of the replies. It makes it much easier to see who’s replied to what, and when. It’s also much more consistent with other systems of threaded discussion.

Finally, I’ve streamlined the interface for ‘actions’ you can take with a post, like editing it, watching it for replies, checking its history, or deleting it if you’re an administrator. By putting these into a drop-down, the actions are within easy reach, but stay out of your way if you just want to read the posts.

Over the next few weeks, I’m going to be working on making the comment workflow AJAXy, making all comments searchable, and other miscellanea.

What do you think of the new interface? How can it be improved? Let me know in the comments.

On June 15, the site-footer and various other messages in the English Wikipedia were changed to reflect the licensing change that the Wikimedia community overwhelmingly approved last month: from the GNU Free Documentation License as the primary content license to the Creative Commons Attribution Share-Alike License (CC-BY-SA). Creative Commons founder Larry Lessig tweeted that it was the “first copyright message ever to bring tears” to his eyes, and Mike Linksvayer called it a “free culture win” in the Creative Commons blog.

A few other Wikimedia wikis and projects have followed in a bottom-up manner, but today we standardized the site language to ensure that all our projects in all languages reflect the new terms (see this message for some more internals about the process). Want to translate text from the Italian to the Spanish Wikipedia? Both are CC-BY-SA. Use content from Wiktionary? It’s CC-BY-SA. A textbook from the French Wikibooks? CC-BY-SA.

Perhaps the most significant reason to choose CC-BY-SA as our primary content license was to be compatible with many of the other admirable endeavors out there to share and develop free knowledge: projects like Citizendium (CC-BY-SA), Google Knol (a mix of CC licenses, including CC-BY and CC-BY-SA), WikiEducator (CC-BY-SA), the Encylcopedia of Earth (CC-BY-SA), the Encyclopedia of the Cosmos (CC-BY-SA), the Encyclopedia of Life (a mix of CC licenses), and many others. These communities have come up with their own rules of engagement, their own models for sharing and aggregating knowledge, but they’re committed to the free dissemination of information. Now this information can flow freely to and from Wikimedia projects, without unnecessary legal boundaries.

This is beginning to happen. A group of English Wikipedia volunteers have created a WikiProject Citizendium Porting, for example, to ensure that high quality information developed by the Citizendium community can be made available through Wikipedia as well, with proper attribution.

The world of free knowledge doesn’t end with Wikipedia, and it shouldn’t. Indeed, license compatibility is just one part of a functioning, decentralized free knowledge ecosystem. Incidentally, with the exception of Google Knol and EOL, all of the aforementioned projects use MediaWiki, the open source collaboration software developed and maintained by the Wikimedia Foundation – so, we are well-positioned to help further develop this ecosystem of knowledge in the future.

Erik Moeller
Deputy Director, Wikimedia Foundation

News and notes: Jackson’s death, new data center, more
Wikipedia in the news: Google News Support, Wired editor plagiarizes Wikipedia, Rohde’s kidnapping, Michael Jackson
Discussion report: Discussion Reports And Miscellaneous Articulations
Features and admins: Approved this week
Technology report: Bugs, Repairs, and Internal Operational News
Arbitration report: The Report on Lengthy Litigation

I am sure that many folks noticed that on the morning of 2009-06-26, techblog.wikimedia.org and blog.wikimedia.org went down.  It turns out that some of the parts of our Wordpress installations were compromised.  I do not want to get in to a direct show and tell of what they did, but hopefully we have hardened the installation to the point that it will not occur again.

This is why the blogs exist on their own server, so when things like this happen we can minimize the impact.  The blogs are both up and running now, along with the other services that were affected.  All but techblog was back online before Friday was over, techblog lagged behind until today.  (As techblog was the point of exploit, we got everything else back up first.)  Other affected services were the Open Conference Systems site for Wikimania 2009, as well as our survey software.  Both of those were back online ASAP after the incident and the rest followed after.

Of course, it was hard to get this information out to folks when the blogs were down!  It goes to show how easily using the blogs to get info out has been, since without it we had to scramble to get the information out of other channels.

Thanks to everyone who assisted in the restoration, and also thanks to everyone for their patience while the system was fixed.

There's an interesting piece in the New York Times today on investigative journalist David Rohde - who was kidnapped in Afghanistan last year and who escaped last week from his captors in Waziristan, in northern Pakistan - and the efforts to extend the media blackout on news of the kidnapping to his Wikipedia article.

The blackout was orchestrated by the New York Times Company and was said to have involved forty international news agencies, from NPR to al-Jazeera. NYT personnel "believed that publicity would raise Mr. Rohde's value to his captors as a bargaining chip and reduce his chance of survival", the story says, quoting Rohde's colleague Michael Moss as saying "I knew from my jihad reporting that the captors would be very quick to get online and assess who he was and what he’d done, what his value to them might be".

Along with staff at other news agencies, NYT personnel contacted Jimmy Wales too, who passed the matter along to a small group of administrators who reverted mentions of the kidnapping and protected the article a number of times over the following months. Michael Moss also apparently edited the article to emphasise Rohde's Pulitzer Prize-winning work on the Srebrenica massacre, as well as his work on Guantanamo Bay, believing that if his captors read the article they might view him as more sympathetic towards Muslims.

Jimbo acknowledges in the NYT piece that the matter was made easier by the lack of reliable sources reporting the kidnapping - a consequence of the blackout - which meant that the biographies of living persons policy could operate to keep any references to the kidnapping out of the article. The policy, of course, was originally intended to keep fabricated material out of articles, but it worked equally well to assist the blackout in this case.

The ethics of the blackout have come into question following Rohde's escape. NPR reported Poynter Institute journalism ethics lecturer Kelly McBride as saying "I find it a little disturbing, because it makes me wonder what else 40 international news organizations have agreed not to tell the public". Dan Murphy at the Christian Science Monitor says that the question of whether the press has a double standard in keeping quiet about their own while regularly reporting on other kidnappings will likely become part of the debate. Greg Mitchell, the editor of industry journal Editor & Publisher, details that organisation's internal debates and ultimate decision to adhere to the blackout. Mitchell raises a potential competing public interest argument, that information about events such as kidnappings in a certain area could, in some cases, help protect the public (though the average NYT reader doesn't hang out near Kabul that often - it might help protect other journalists though).

On the Wikipedia front, this is an interesting biographies of living persons case because every aspect of it involves journalists, who as a profession develop, apply and teach a whole suite of ethical principles governing their work, principles that many have suggested Wikipedia ought to adapt or learn from.

It's regularly true that hard cases make bad policy, and it is so here: the kidnapping was said to have been reported by an unnamed Afghani news agency, and apparently by Italian agency Adnkronos too; the existence of reliable sources on the matter (which I cannot verify due to absent or broken links) throws into doubt the legitimacy of enforcing the blackout on Wikipedia.

This may well put a wedge between two similar but distinct camps of support for the biographies of living persons policy: those who believe that such articles should be written from a "do no harm" perspective, and those who have a similar sympathy but only go so far as supporting a strict, immediatist adherence to ordinary content policy (instead of the typical eventualist stance), and no further.

Lately I've noticed a several people around Wikipedia whose only purpose seems to be to criticise and take offence at every single thing that goes on. These people range from new editors, to admins who have been here for years.

For example: there are numerous editors with a ridiculous idea that admins are out to "get them". To them, Wikipedia is an online battle: the fight between the editors and IRC admins... utter nonsense. It is certainly childish playground mentality at best. There are no gangs, for goodness sake.

There are other editors whose purpose is to act like some sort of martyr amongst their friends. Deliberate trolling is stuck up for under the guise of "opinions are allowed", and when the troll is blocked, the troll is made out to be a hero. Madness.

Then there's the infamous requests for adminship page. This is the best place to find negative people - just look at the oppose column on any random RFA! OK, not all opposes are dumb. I actually saw a fairly decent oppose today, based on age of all things. It's fine to oppose people - I've done it myself, numerous times. The problem is, the ideology several people have. They actually come to the RFA looking for reasons to say "no". Unbelievable - it's not like we have a limit on how many admins we can have. What's wrong with being positive, and looking for reasons to say yes? I simply can't understand the negativity of some people. And then, there's the opposes themselves. Let's face it, many opposes are scraping the barrel. But when such opposes are as blunt as "No frickin' way", we have problems. What an insulting way to say no. How completely unnecessary. Yet, people do it.

The negative people on Wikipedia are thankfully a minority. Most of us edit for fun, and have a good time doing it. We don't go looking for fights, or enemies to make, and we get on writing articles or whatnot. I hope these negative people aren't as angry and sour in real life.

The last three days I have been to OTT'09, the open translation tools 2009. It has been interesting so far. Given that it is about OPEN translation tools, the public is quite amazing. You have people who are about content, people who professionally translate, people who localise software and people who write tools to support all that.

One of the people I am always happy to meet is Dwayne Bailey of translate.org.za. Dwayne was leading a session on translation and localisation in Africa and for me it confirmed many of the things I think I know but it brought also a new opportunity. I learned about ANLOC, the African network for localisation, one of the things I find awesome is that they have defined requirements for a minimal support for African languages.

The requirements can be on many levels and the cool thing is that with these requirements a website can be assessed. I would really welcome Wikipedia or better even MediaWiki to be assessed.
Thanks,
GerardM

With the news that Wikipedia is going to provide support for video, I am sure that a wealth of information will become available in this way. As video is created everywhere, much of it will not be understood because of it being in this language that you do not understand.

At the OTT'09, the open translation tools conference, I met Ed Zad of dotSUB. DotSUB is a project where people add subtitles to video. The functionality of dotSUB is quite important and I wonder how we will do this for the content for Wikimedia projects. The cool thing is that at dotSUB they are considering making their cool functionality open source.

Ed was an inspiration at this conference; he has a long career in translation and he brought the perspective of the translation industry to this conference. For me it gave this conference the depth that makes this a wonderful conference.
Thanks,
     GerardM

The 1st Wiki-Conference New York will be held over the weekend of July 25-26 2009 (confirmed!) at New York University, and hosted by Free Culture @ NYU and Wikimedia New York City.

Wikipedia founder Jimmy Wales will be giving a keynote, and we will also have a second keynote speaker TBA.

Since this is an unconference, participants are encouraged to give your own ideas for topic sessions!

This is also a great opportunity for people who aren't so wiki-savvy to get a fun introduction. All are most certainly welcome.

Oh yeah, and we're having the Central Park picnic again too!

Register here:

http://bit.ly/wikiny

When several different people make queries and the same advice needs to be given, it's probably better (or at least less work) to blog the darn thing. So here's one of my little methods for defusing content disputes. Call it the three week rule.

This is one of the reasons why the cases I take to formal dispute resolution almost never relate to my own content disputes.

First, a little harmonious spirit.

Now here's the trick: if you've tried the bold/revert/discuss model and it isn't going anywhere useful, then consider this: walk away for a while. Give it about three weeks. Hang out somewhere else. Chill. Wikipedia has millions of other articles.

Sometimes the person you were locking horns with ain't so bad. In three weeks, if that person has a broader set of references and perspectives to bring to the page then that's plenty of time for them to shine. You might be pleasantly surprised.

Second option: your hunch is correct and that dude's a POV pusher. Let them own the article for three weeks. If that's what they really are then they'll slant the article even further so it's obvious to everyone. Once things reach that stage the problem is easier to correct.

Third option: the individual is a troll (or at least feeds off conflict). So stop acting like an immovable object, and watch the irresistible force wander elsewhere. There's a beauty to the Zen approach.

So give it three weeks. Let the article be wrong. When you return, post politely to the talk page. If nobody objects then go ahead and edit. If somebody does object then don't quarrel; open a content request for comment promptly. If you're really right then uninvolved editors will step forward and agree.

It's surprising how often this turns out well.

----
After

So, we had a major embarrassment last night. It consisted of multiple factors:

• We don’t have parallelism coordinator for our most cpu-intensive task at Wikipedia, so it can work on same job in ten, hundred, thousand threads across the cluster at the same time.
• Some parts of our parsing process ended up extremely CPU-intensive, and that happened not in our code, but in ‘templates’, that are in user-space. We don’t have profiling for templates, so we can just guess which one is slow, which one is fast, nor their overall aggregates.
• Some parts of pages are extremely template-heavy, making page rendering cost a lot (e.g. citations – see this discussion).
• In order to avoid content integrity race conditions, editing process releases locks and invalidates objects early, separated from ‘virgin parse’ which populates caches.
• It takes quite some time to refill the cache, as rendering is CPU-bound for quite a while in certain cases.
• During that short time when caches are empty, stampede of users on single article causes lots of redundant work across the cluster/grid/cloud.
• Michael Jackson article on English Wikipedia alone had a million views in one hour

So, in summary, we had havoc in our cluster because stampede of heavy requests between cache purge and cache population was consuming all available CPU resources, mostly working on rendering references section on Michael Jackson article.

Oh well, quick operations hack looked like this:

Index: ParserCache.php
===================================================================
--- ParserCache.php	(revision 52088)
+++ ParserCache.php	(working copy)
@@ -63,6 +63,7 @@
  if ( is_object( $value ) ) {
    wfDebug( "Found.\n" );
    # Delete if article has changed since the cache was made
    // temp hack!
+   if( $article->mTitle->getPrefixedText() != 'Michael Jackson' ) {
    $canCache = $article->checkTouched();
    $cacheTime = $value->getCacheTime();
    $touched = $article->mTouched;

It is embarrassing, as actual pageview count was way below our usual capacity, whenever we have problems is because of some narrow expensive problem, not because of overall unavoidable resource shortage. We can afford much more edits, much more pageviews. We could have handled this load way better if our users wouldn’t be creating complex logic in articles. We could have handled this way better, if we had more aggressive redundant job elimination.

Thats the real story of operations, though headlines like “High profile event brought down Wikipedia” may sound nice, the real story is “shit happens”.

Our CTO Brion Vibber offered a fascinating post on the Foundation’s Tech Blog today, highlighting the incredible traffic spike and related problems caused by the news of singer Michael Jackson’s reported death.

Expect the tech blog to be updated as other server developments unfold, and of course the Wikipedia article to go through some fascinating evolution and discussion.

Jay Walsh, Communications

News agencies today are reporting that pop star Michael Jackson has been hospitalized, and perhaps died. We can all think back on how the King of Pop has touched our lives, but today we can also see how high-profile news events can affect a web site… See also past events such as the Popedotting and the 2008 US election.

Here at the office we first noticed something was going on when IM services such as AOL Instant Messenger started logging people out — we quickly noticed that our own servers were hitting load spikes, and suspected there was something going on…

Server CPU load spike (likely several more to come):

The actual traffic load spike is subtler; server effects can be disproportionate to the actual traffic:

Update 22:53 UTC:

The traffic is pretty much holding steady but we’ve still been seeing intermittent load spikes:

These are at least in part due to one of our memcached internal data cache servers going wonky and swapping due to overuse of memory from text storage running on the same node. We’ve reduced traffic on the node and restarted it to even out its memory usage. (Thanks Domas!)

Update 23:00 UTC:

You may see intermittent messages like “(Cannot contact the database server: Unknown error (10.0.6.24))” as temporary database overloads cascade around the system. Sorry for the inconvenience while we work the kinks out; just wait a few minutes and try again…

Update 23:43 UTC:

We believe a large chunk of the CPU overload is due to cache swarming — many visitors simultaneously causing a re-render of the page due to an expired cache version. I’ve put in a temporary hack which will reduce the amount of rendering, but may cause some people to see out of date copies of the page.

Update 2009-07-02:

Here’s a link to Domas’s blog post with technical details on the cash swarming problem.