mysql update in RHEL5v4

Cool, it looks like there’s an update to mysql in RHEL5v4 (to version 5.0.77 )

That means this bug is about to get fixed. I wasted a bit too much of my life on that bug last year.

p.s. 5077 sounded awfully familiar – but after double-checking, it was not the unit number for the famous fictional MASH unit. Whew.

How Project Names Happen

So, being a developer and the systems manager hath its privileges. I get to wade through a convoluted process to come up with names of projects – and I get to apply them. Because, well, I can.

I’ve already rambled on more than once in the blog about desktop and server names, but I’ve never really talked much about coding projects.

Really, mainly that’s because the names are usually boring. And also that I/we thankfully don’t start many coding projects. Starting an application/project is not something to be taken lightly – and well, we already have what is likely too many projects anyway.

However, today I am taking the baby steps into a new rails project. Currently – our “Identity” project is serving as a hub of activity tracking. For a whole host of reasons, I’m thinking of separating the actual data collection and storage and “generation” of that data into a separate application (but leaving some of the views into that data within the Identity tool).

So, what to name it? We’ve named projects after Greek Words, the protagonist from a movie about a talking plant with psychological issues, and boring things like “project function.”

But this needed different. Our activity views are part of the idea of freeranging or “work wide open”. And after spending a few minutes in Google and Wikipedia trying to come up with names that were related to “free-range” and getting no further than pasture management strategies – I needed a new tack.

Our activity views are both a reporting/monitoring mechanism – but they are also an experiment in workstreaming.

Hmmm…. streams. The first free-associated “stream” idea that I had was clearly NOT going to do.

So what’s the next most famous “streaming” idea? That’s right…

Dr. Egon Spengler: There’s something very important I forgot to tell you.
Dr. Peter Venkman: What?
Dr. Egon Spengler: Don’t cross the streams.
Dr. Peter Venkman: Why?
Dr. Egon Spengler: It would be bad.
Dr. Peter Venkman: I’m a little fuzzy on the whole “good/bad” thing here. What do you mean, “bad”?
Dr. Egon Spengler: Try to imagine all life as you know it stopping instantaneously and every molecule in your body exploding at the speed of light.
Dr. Ray Stantz: Total protonic reversal!
Dr. Peter Venkman: That’s bad. Okay. All right, important safety tip. Thanks, Egon.

Hence, the new project for activity tracking, Google visualization, json data feeds, etc.

Protonic Reversal

And except for the ones brought to you by the stork, and the tooth fairy – is how project names happen. I mean, really, who you going to call?

(okay, sorry, couldn’t resist)

It’s about the details

My default feed yesterday switch back to RSS 2 from Atom (and a huge thanks to Sam for pointing this out) because of a upgrade to WordPress 2.3.1.

It was another reminder of how things fail because of a cascade of missed details.

The immediate detail, was that I completely forgot my own quick hack to WordPress 2.3 to force the Atom default. That’s the danger in making “one-off” changes to software that you use. Quick one-offs don’t become part of one’s standard workflow, they are too easily missed later, and upgrades will usually wipe out one’s changes. The old axiom applies here – “just because you can, doesn’t mean you do, because it will bite you in the rear later”

If I’m going to make changes like this, I probably should do what I do with MediaWiki at work, and pull things from Subversion and merge them into my own trunk. But that’s just flat out silly for a few lines of changed code.

The second mistake is that I should have – at the least – taken Mark Pilgrim’s advice and put together the two liner plugin to remove the rss2 action.

That’s just complete laziness on my part. I had it working and I didn’t bother it again – I wanted to spend more time investigating how the whole “actions” thing works in WordPress, and had grand dreams of extending that idea into a plugin that could be configured from WordPress itself. Then I decided I’d rather play with the dog.

There’s this continuum of solving problems that starts with hacks and goes to solutions. And the problem is, solutions take a lot of work. So the happy medium is somewhere in between. My change was hack that bit me in the rear later. Mark’s change is something that will likely work across upgrades. And that’s what I should have done right then, and I absolutely knew better than to continue with my own podunk modification. There’s another axiom there “just because it works, doesn’t mean it’s right, especially if isn’t going to work later”

So, the default feed format changing is completely my fault. If I care, and I do, I have to make sure that the things I really care about keep working. There’s an ongoing maintenance cost to my commitment.

But the third detail in this, not that it excuses my mistakes in not being diligent about my details, is that this really should be part of the core of WordPress. It really doesn’t make sense that the Atom feeds are there now, but the software is so bent on making RSS2 the default and not letting folks change it easily. There’s really no excuse going forward, especially given the patches available.

(and yes, it works just fine, as my own patch -p0 and plugin drop-in can attest to – that I’ll have to do again for 2.3.2 – Mark’s plugin is there as a backup this time

I’ve been on the WordPress dev’s case about this a long time. Always in this blog, and I’ve never put code where my mouth is. I should have put up or shut up. Sam and Mark, and dozens of others thankfully have, with WordPress, and in dozens of other places that matter.

But even if I should have shut up already – there is something core that I’ll repeat here. I know that the users don’t care. I know that it doesn’t make a dang bit of difference to Aunt Millie whether things are RSS 2.0 or Atom. But RSS 2.0 is the IE 6 of data interchange. WordPress’s continued traction on making it easier to allow the software’s users to make the Atom feed the default is like continuing to excuse IE 6. Sure, we have to put up with it, but we don’t have to keep making it hard to choose other things.

As developers, we have the ultimate responsibility to make sure that we are laying the foundation for how we ship data around – now, and 5 years from now. I sure hope that the WordPress core devs run with the patch this time. It really does matter. You’ll still have a successful business model without it, but don’t make us continue routing around the damage.

[ed. note - I read this again after Sam added his second paragraph about the patch and my own post was a little unclear, so it was edited slightly for my actual intent. While I don't think that RSS2 should be the default - I get not making the change right now, just stop making it hard to make a choice with the feed format]

Peeling the Onion

And no, I don’t mean The Onion – which would have been far more entertaining.

Through Joi Ito’s blog I have recently become aware of the phrase “Yak Shaving.” Joi wrote about it in 2005, here’s WikiPedia’s take – and here’s an etymology from Alexandra Samuel.

When I first read Joi’s blog. I took Yak Shaving to mean a pointless activity (Joi writes in a bit more layered fashion than most folks). It’s part that of course (read Alexandra’s post). But it’s more about good problem solving. Especially when you go and read the entry in the Jargon File

Any seemingly pointless activity which is actually necessary to solve a problem which solves a problem which, several levels of recursion later, solves the real problem you’re working on.

One of the things that I’ve spent my entire career doing is looking for the root causes of problems. Yes, just like every other system administrator/developer, there are times that I’ll defer the problem to another day (to this day, I’m still avoiding a mime types/magic file problem with Red Hat and PHP and MediaWiki that I’ve spent too much time on already). But I recognized a long time ago that digging into something, rather than stopping when the problem was mitigated was going to be much better for everyone. I spent a lot of long nights doing this early on, and still do occasionally – and I’m thankful for some early examples from mentors that encouraged this. It’s made me a much, much better troubleshooter over the years for doing so.

The latest peeling the onion activity came last thursday. I arrived at work, with every intent of taking the example OpenID rails server in the ruby-openid distribution and beginning to pull it in our internal authentication tool. Doing that is very much a “Yak Shaving” activity. There’s some other more pressing problems, but doing OpenID in the internal application solves part of 2 or 3 other problems.

Well that fell by the wayside by mid-morning. We have a preview version of our public website. Most days I’m actually surprised that it works given our Rube-Goldberg esque technical framework for taking mediawiki pages and getting them to the public web site. But it’s been a real benefit internally to have the preview site. Making it happen also made the public application more flexible too.

Well, mid-morning thursday, there was a report that content wasn’t updating on the preview site. At first it was thought this might be a by-product of the previous day’s activity – pulling out a rewrite rule that was eating encoded “?” characters (i.e. %3F) in MediaWiki page titles and causing 404′s by the time those URL links made it into a Flash application and a Rails application. In the process of fixing that, we actually fixed another problem where the source URL for each page in our atom update feed was wrong.

Making that URL correct was what broke things the next day. It turns out that Problem #1 was that the process that pulled in the atom feed keyed in on the URL as the unique identifier for the content (as a fake unique identifier actually, it wasn’t enforced by MySQL). Since the URLs changed when they were fixed – voila! duplicate content – and of course the Find was doing a Find first – and pulling the original, un-updated article.

There was a whole lot of scratching our heads (okay, there was a lot of cursing) about that unique key. The URLs in question are “internal” and pretty much guaranteed to change. Keying off that certainly wasn’t great design. I guess it goes back to that original issue – it solved the problem for a day, but no one gave any future thought as to what it would impact.

So we needed to key off the page title. Well, the page titles weren’t unique either. Which was also a head scratching/cursing problem. MediaWiki page titles are unique in a namespace, and our find functions assume they’ll be unique as imported, but that uniqueness was not enforced.

Well, MySQL folks can guess what happened next. We’ve never actually ever dealt with the collation issues with our MySQL server (there’s a lot we haven’t dealt with with our MySQL server – but that’s another post for another day).

For good or for bad, I really didn’t understand why our collation was set to “latin1_swedish_ci” – and thought that I had made a major mistake setting up the server defaults in the first place, that no dev ever caught when thinking about their schemas. I was pretty relieved to find out that that was just the default for MySQL in the first place.

James’ absolutely groaner of a quote?

Well at least we didn’t bork it up

Well, MediaWiki titles are case sensitive, and it made sense for that column to be case sensitive too – so in went the migration. This actually gave the additional benefit that searches for articles titles would actually be accurate now (even though we have some content that differs only in case that needs to be fixed).

execute "DROP INDEX title_idx ON wiki_articles"
execute "alter table wiki_articles change title title text character set latin1 collate latin1_general_cs null default null"
execute "alter table wiki_articles add unique title_idx (title(255))"

(p.s.
select somecolumn,count(*) as n
from table
group by somecolumn
having n > 1
is a wonderful tool to stick in the tool belt)

After all this is done, we had to import the content again. It’s about 25MB of an atom file – 5,000+ pages of content dating back to last september. Our standard processes of trying to pull this data in with a HTTP GET takes too long to run with the HTTP timeouts in the libraries we use – so a long time ago I modified our code to read the data from a file when needed.

Well, when a contractor modified the code to stop using the FeedTools library and just do some simplified parsing for our internal purposes they took out the “read from file” functionality and didn’t replace it. Which generated some more head scratching and cursing. So that had to go back in to get all the content back into and corrected.

A simple problem of content not being updated highlighted 4 different problems: wrong key for the content, no unique enforcement for any keys, wrong collation on the column, and data import from files missing.

We could have stopped early on by just killing the duplicated content with the wrong URL, updating it, and reimporting the latest changes. But we didn’t. The application we fixed didn’t matter for public use – but our fixes prevented some future problems.

I guess we shaved a few yaks that day. And proved yet again how important it is to get to the root of problems. And how painful it is later when you have to go back in behind yourself and others because it wasn’t done originally.

Designing Content for the Web

Shelly Powers resurrected a post a few days ago on javascript “widgets” (without much thought you could extend this to any blob of javascript doing http requests and fun little rendering things with local and server data without refreshing the page).

It’s a long piece, that I think boils down to “web designers should use them responsibly”

I’m going to extend this in another way. Shelly said something in particular that I want to point out:

The same person who wrote the comment about widgets also mentioned how browsers load top to bottom, left to right. It’s been a long established policy to make your web pages more accessible to screen readers by putting the changing material first in the page layout, and then the more static elements. In a weblog, this means putting the content first, and then the sidebars. The bars may appear to the left, but in the actual physical layout design, they should physically exist after the content, so that the content is always first.

You can’t design content for the web unless you know how the web works. I mean at least at a high level – about the fact that browser software uses HTTP to request data, web server(s) return that data, and the browser software is responsible for beginning to parse that data as soon as it arrives back from the web server(s), and follow its instructions – either make more requests, start rendering text and graphics in the data, or start running code embedded in the data in whatever code parser/compiler the browser supports. And add on top of that how other software (xml clients/”feed readers”, embedded browsers, screen readers, the code that browsers execute, other web servers even) also will request your data and follow its instructions.

Well, I mean you really can design content for the web without knowing any of this – but if and only if you trust the guidance of the people that do.

[Updated: The more I read and re-read my original post, the more I realized that it was one big long "duh" So I just decided to edit the thing and get to the heart of the point I wanted to make.]

wp-atom and wpLicense

Thanks to a couple of line changes in the wpLicense plugin – namely after line 384 – adding:

add_action('atom10_head', 'cc_atom_head');

and because of what is likely an output buffering issue somewhere – I had to change line 84 from:

link rel="license" type="text/html" href="'.licenseUri().'"

to:

link rel="license" type="text/html" href="'.get_option('cc_content_license_uri').'"

(yes, I need to debug this – it’s kind of a hack)

I now have the wpLicense plugin working with Benjamin Smedberg’s atom 1.0 plugin

All this so that Conversations with Plastic Dinosaurs is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 license.

Quote of the Day

From me in email:

run-of-the-mill developers fix things
great developers fix things and share those fixes with their colleagues

I don’t do this as often as I should, but by damn I’m going to get better at it. There’s absolutely no excuse for making a fix, and not being aware that your fix might be useful for other tools that your colleagues work on.

Wow They Did It

This isn’t quite on the scale of Linden Labs opening the sim code (actually, I’d be far more in favor of them packaging it up so that you could run your own sim) – but I’ll be damned – they open-sourced the client

That will certainly reverberate across the 2nd life and other 3D virtual world blogs today.

There’s a whole lot we just don’t know

This article from David Polack is fascinating. (via another interesting article from Jeff Moore about OOP, PHP, and futuring).

Written back in October 2006, after RubyConf – it talks about the language VM’s under development for Ruby – and well, it’s not all that positive on the future of Ruby for depolyment (which of course makes big enterprise-oriented shops with large development staffs doing things like payroll nervous).

Did I say it was fascinating? So are the comments. This is a space that most of in the ranks of small shops are nowhere near – in fact it’s usually over our heads most days (maybe every day, usually the small shops are just dealing with people that want some checkbox to move from one side of the screen to the other, or doing the web application support equivalent of helping someone “print from Word”). I’ve been more than a little bemused recently with all the mentions of JRuby (like why the heck do I care whether Ruby compiled to bytecode and executed by a JVM). I get the implications when I stop and think about for a bit, but I don’t really care.

It’s damn sure completely off the radar for the folks that use tools produced by the languages like ruby, or java, or php, or haskell, or whatever language you want to pick. I absolutely know that watching this kind of stuff is not understood one whit by a lot of the people in “technology.”

Anyway, I recommend reading it, not because you’ll care, but it does impact the future of the tools we all use. And it will show you a little, of just how broad this area called “IT” really is.

I like ruby as a language a lot. I like it better for my systems purposes than perl, and php. From a web development perspective – I think it’s a wash vis-a-vis the language itself – though I enjoy ruby more than I enjoyed php. Rails was/is a big deal – and sure makes a lot of what I’ve done with PHP before a lot easier to do (at first, and then it’s just good old programming sweat after the first few days) Maybe it’s not any better or worse than something like Django (but I certainly like ruby better than what little python I’ve done) but it’s better than anything PHP had (though I’m completely ignorant with any current PHP frameworks).

I don’t particularly care about enterprise computing either. I’m not sure that the enterprise IT space ever actually helps anybody do anything interesting (but it does make sure they get paid, which is pretty damn important). But the enterprise drives a lot of things and it will be a interesting 2 or 3 years watching all this shake out. What else don’t we know that will come up and impact our business?

Almost, but not quite, completely off-topic

Articles like this one from the ADC reference library (I subscribe to the feed) – make me so completely happy that I’m not doing desktop programming anymore (I never have done it on the Macintosh, Win32 was enough).

If you receive this return value from AuthorizationCreateFromExternalForm it means that the AuthorizationRef from which the AuthorizationExternalForm was created is no longer valid. Typically this is because the process associated with that AuthorizationRef has quit.

wow, I just fell asleep at the keyboard. (for my estimated 6.5 readers, it’s not safe to wake up yet from my last post)