Here Comes the Sysadmin

To be titled “Rambles of an University System Administrator” – my professional blog sure has suffered from a dearth of system administration topics.

Well, no more.

eXtension finally has some server equipment – and after a few days of crawling around on my hands and knees racking it all up, I have several observations coming. Now I realize that it’s somewhat bad form to just stick a post here to say what I’m going to say. But that’s what I’m saying.

Meanwhile, here – remember back to “Here comes the Hotstepper” and get it stuck in your head while you wait 🙂

_Hit It! Nah, na na na nah, na na na nah, na na nah, na na nah, na na na nah

Here comes the sysadmin, configure-er I’m the macintosh gangster, configure-er Pick up the news in-a de ‘gator, configure-er Still love that rss, configure-er

No no blogs don’t die Yes they mul-ti-ply Unit test we’ll hear argument sing Act like you know Scoble I don’t know what Google don’t know Touch Apache up and go, uh-oh! Ch-ch-change change logs_

Okay, it needs some work – but it’s not bad for five minutes of work.

(yes, I know I said professional blog earlier, but it’s all relative)

Something new for me and ewe

This post marks the close of a chapter in my working career.

For the last several years, I’ve had an on-again, off-again relationship with several thousand lines of PHP code that has served as a personal wiki/weblog tool. I started it as a personal project in 2001, storing and indexing lots of little text files because I wasn’t yet convinced that largely static web delivery should require yet another server process like a database. I used it for personal hosting – and the lessons learned writing it (and a the realization that I needed to get with it and join the rest of the world using MySQL for web applications) and many hundreds of fitfully scattered hours later it became colloquially “ewe” for the “eos web editor” that ended being used, well, imposed, on several sites for the College of Engineering at NC State.

Like any piece of software that grows out of a hobby and a mission, ewe represented more than a wiki/weblog. I’m sure that Dave Winer would say the same about Radio. Or that Noah Grey would say about Greymatter. Or the Michel Valdrighi would say about b2/cafelog (the software that became WordPress). Not that I or ewe are really in same categories as those, but having watched all of those efforts over the years, there are some shared characteristics – good and bad.

Like a lot of things that I do, ewe was a attempt to wound several flying animals with a single projectile. At face value, It was an learning exercise to learn both PHP and MySQL, and how to develop in the constraints of the Engineering web environment (and to support development within those constraints). It was a way to try and encourage the College of Engineering system administrators to share and collaboratively edit IT documentation. It was a way to get the ITECS Systems group to publish out what we were doing. And it likely was a way to stroke my own ego with a professional weblog.

Deeper down – though it sounds like complete b.s. – ewe was most of all a manifestation of a hope to improve the levels of collaboration and communication in the College of Engineering – and even the campus at large. Unfortunately, my dreams and hopes are bigger than my ability (and time) to develop software.

But I gave it the “college try” – and I piddled away on what became ewe. But now that I’d like to run it on a host that’s not part of Engineering’s infrastructure, I need to give it a new authentication tool (e.g. a username/password table, the form to login, the encryption to protect the password transfer, the code to write the cookie for successful login, yadda, yadda, yadda) – and I want to fix permalinks, and I want to incorporate XMLRPC publishing (or Atom publishing), and I want to clean up the Admin interfaces, and, and, and…. None of which is particularly hard – but it’s all quite time consuming. And when I complete all that – it’s just going to be for my weblog. The sites that are currently using ewe aren’t really going to want to migrate, because try as I might to make that really clean, something is going to change for the people using it. And the sites that are using it aren’t really going to grow, because some of the limitations need to be fixed. So it’s a bit chicken-and-egg with my time in the middle of it.

Add to that, that I have a lot of system administration to do. I’m not a developer, just a hacker (or maybe just a hack*), and I have other things to hack.

Hence, WordPress. Which while I have some bias with my own code that mine is a whole lot nice internally – WordPress looks great from an administrative perspective. And it works. And there’s a community around it. And I can finally post from Marsedit or ecto. I just picked a temporary theme (albeit a nice one) – and I’ll hack together my own theme soon. The ego stroking lives on.

In the end, ewe served its purpose well. I’m not so sure it got people collaborating any more – but it did facilitate at least a modicum of communication. And working with it let some students pad their resumes and get jobs. And I learned a little here and there about PHP. And those are pretty good things all told.

So welcome to the new rambles, same as the old rambles.

(* it’s at least a 32-bit hack though, as opposed to a two-bit hack)

Looking for a good Linux Sysadmin

So, hopefully in the next few days – eXtension is going to be hiring a System Administrator as a SPA position (an Operation and Systems Analyst in the IT Career Banding system) at NC State University. The position is grant-funded so it is “time-limited” – meaning that there’s a possibility that the grant funding could go away in two years – and employees that are in time-limited positions aren’t eligible for severance pay or North Carolina Reduction-In-Force (“RIF”) status (we are pretty confident that the funding is going to extend well beyond 2007).

The job description:

65%
Maintain the Linux, Windows, and OSX, database, web, source code control, mail, authentication, and messaging servers within the eXtension initiative, both at NCSU and at partner sites, troubleshooting and resolving technical problems as they arise, and keeping all software versions up to date. Implement and install new servers and service software for eXtension projects.

15%
Keep up with the latest security threats and vulnerabilities; communicate with eXtension staff on problems and issues that may affect eXtension services and those of extension partners through the country. Provide consulting and contact services to pull in other land-grant system administrators where needed.

10%
Work with other eXtension staff and land-grant system administrators in building, debugging, and testing services and tools created for eXtension. Provide programming and scripting services to help integrate those services into the larger eXtension system.

10%
Test and evaluate new server and service technologies in support of the database, web, source code control, mail, authentication, and messaging. Provide consulting for other system administrators in the land-grant system.

Basically, a lot of OS configuration, Apache installation, and installation of various web apps and then other things like mail, jabber, subversion, etc. The position should allow the person to participate in a few community projects that interest them, and there is a good hardware budget, and a good training/travel budget.

You’d have to ask former employees of mine, but I hope that I’m an OK fellow to work for. I’m a perfectionist at times, and more than a little anal about communication, checking logs, keeping configurations up-to-date, security, and trying to “do it right, rather than right now” and tend to expect the same, but I try to keep all in check and balanced with reality 🙂 Family comes first, and strong ethics and strong morals and wanting a fun workplace are real important to me, both in myself, and in the person that I’d hire. Having independent thinkers that aren’t afraid to question the status quo is important and encouraged too. Both Dr. Gamble (my boss) and I are oriented to open source/standards/and related things like creative commons philosophies, both using open software/information and creating it (although we are Macintosh desktop weenies).

This job is part of the Information Technology and Application Development team for the National eXtension Initiative. The goal of this initiative is to plan and implement a national web-based information and education network for current and new Extension clientele. By doing business in a new way, it will expand learning choices and methodologies in support of just-in-time learning by providing coordinated access to objective science-based information of land-grant universities. We strongly believe in the mission of the land grant university and cooperative extension.

The upside is the University working environment, the downside is that salaries are likely a little lower than industry average, and there’s basically little or no relocation assistance (there are discounts with a local realtor company). But if you or someone you know would be interested, have them drop me a line. Applicants will need to go through the official University application process once the position is posted. And I’ll update this post with a link to that once it’s up and posted at the NCSU jobs site

Pretty Pictures

Well, thanks to a comment from A. J. at UT-Knoxville – I know that my RHEL/Fedora problems under Virtual PC 7.0 are not only my own.

I have gotten Ubuntu installed. Only to have the X Configuration go wacky and produce screens that look more like fractals:

than Gnome desktops. I’ve played with the /etc/X11/xorg.conf to no avail at the moment. Great abstract art generator though.

But hey – I’m further along there! (next I’ll try suse)

[update] – Well, while I edited my xorg.conf file to remove the option to do 24 bit color, I failed to changed the “DefaultDepth” parameter (I think it’s a tad silly that you can set the DefaultDepth to an option not in the file, but that’s just me). Changing that gives me a rather happy ubuntu install. I was reminded to look at this from the ubuntu wiki.

For the record, the OpenSuse install appears to completely fail, but the only thing I tried there was booting off the cd – I’m not sure my cd image is even good.

Fink can be unfun

Installing Apache and PHP with Fink can be seriously un-fun, as Rafe Colburn, my new colleague at eXtension, discovered yesterday

In fact, after writing up instructions for using fink to do an apache+php install – I seriously wondered what crack I was smoking. I still stick by the fink install, if only to catch up on the soaps while you wait for it to compile, fail, and compile some more.

Thankfully for the “I just want it to work now and not dork with this mess” crowd – Apple supplies a perfectly good Apache and PHP install. And MacDevCenter has some great instructions for using that built-in Apache and PHP – starting here and continuing on through here

The masochists probably want to stick with the Fink install – especially once I get around to documenting the joy that is generating self-signed SSL certificates 🙂

Virtually RHEL

Every attempt to try and install RHEL WS 4.0 (the University has a site license) that I’ve tried with a ISO boot cd has resulted in a “An unrecoverable processor error has been encountered” message.

This occurs when the linux boot image tries to load the kernel.

At the moment, given that I don’t know enough to get more detailed troubleshooting – either out of the RHEL WS boot cd or virtual PC (7.02) – I can only conclude that it’s not possible to install RHEL WS 4 under Virtual PC 7.02 for the Macintosh.

However, Fedora Core 4 is happily – though really, really sloooooowly installing as I type this.

[update]: I spoke too soon, Fedora Core installed, but also produces the above error. Sigh.

This does not bode well

So this morning I download the new Backup version 3 application that comes with my .Mac account. I use Backup at home, and it works – but only if you never let the Backup log get too big (like more than a few days).

I’m hoping that Backup 3 fixes that – and wanted to see what it looked like in the office.

Somehow a blank “keychain access” dialog doesn’t bode well

[update]: for the record, running it the second time resulted in a keychain access dialog with the text and buttons that you’d expect in it. It’s more superstition than anything else 🙂

Universities and Personal Information

It looks like Miami University (Ohio) had a lot of SSN and student data on the web for several years. The great thing is that at least they didn’t issue a press release blaming everyone else for exposing information too.

As ITECS Systems Manager I completely reorganized our web presence – almost completely driven by the fact that our “web servers” would point to the top of the AFS filespace to deliver files – meaning that anything that had open permissions ended up in Google – which meant a lot of SSN spreadsheets where professors had been posting grades. It was a design failure perpetuated by the IT staff that brought up webservers in a different era of information exchange – but they never changed as the world changed.

I took a fair amount of flak for this – the faculty were upset with me for making them make changes, for a while, my team members were bemused at my entreaties that this was a severe problem AND HAD TO BE FIXED, and everyone was worried about the support load. My IT peers that had similar environments didn’t change for several years after because they were afraid of this support problem.

I had my share of screwups. I made decisions in IT that weren’t always the best decisions, and my users and peers could have legitimate complaints about things my team did – or more often – didn’t do.

But by golly, I got that web change right.