Openfiler progress - pump up the volumes

I have been making some progress with the update to Openfiler. So far I have the basic authentication happening, but haven't really touched anything to do with LDAP or AD/Samba integration. System date/time needed a bit of work to overcome the fact that 'timeconfig' cli no longer exists. Networking didn't need much and seems to be reasonably solid. Similarly with the rest of the basic system administration.

Most of my time has been taken up with getting the volume management working. First thing was to get it to even see the disks. 'list_disks.pl' relies on 'parted print-fdisk' to get a list of the disks and their partitions, but the 'print-fdisk' option is another Openfiler-ism. It was a simple case to replace this with a call to 'fdisk -l', which gives the same information.

By far the most effort has gone into getting the RAID configuration working. Once again, Openfiler utilised a patched version of a standard utility - in this case mdadm - to provide output in an Openfiler-specific format.  Since I am trying to avoid this, I had to do some digging into where the information is used within Openfiler, which included digging into the original source of mdadm to understand exactly what standard mdadm outputs vs what the Openfiler patch does, and then working out how to do the same without a patch. I started down the same path as I had previously with 'authconfig', but now realise that was a bit of a hack.  In the end I have put the code where it should be, in 'md.inc', which I find a far more satisfying solution.  In the process, my understanding of PHP has grown measurably. This has given me the impetus to go back and do the same for 'authconfig' but I will probably leave that to a later time.

Having gotten RAID and LVM workable, at least for simple 'volumes with file-systems', I now need to tackle iSCSI.  The original Openfiler uses the iSCSI Enterprise Target implementation, with it's command set scattered throughout the product. The standard iSCSI that comes with CentOS 6 is the Linux SCSI Target Framework with a completely different command set and configuration (which has already been superceded in the standard kernel by the LIO framework). SCST is also another viable contender with a lot of support. I could take the easy way and simply build and install IET, but that wont necessarily be a long term option (and, once again, means it will be one more thing 'out-of-band'). So it looks like I will be doing a lot of 'find-and-replace' but at the same time, I will think about re-architecting to maybe make iSCSI a 'plugin' where I will come up with a generic API, and the implementation will be done in include files that are chosen at install time.

After that, I think the next major task is sorting out services, particularly managing both 'init.d' based and xinetd-based services.

Printing authconfig output as XML

Just a quick followup to my post about bringing Openfiler up-to-date.  Here is a script which takes the output of 'authconfig --test' and formats it as XML.

#!/bin/bash
authconfig $* --test | sed 's/ is \| are /=/
s/ = /=/
s/ by default\|always \| (.*)//
y/ +/__/
s/^_/ /
s/_or_.*=/=/
/="/! s/=/="/
/"$/! s/$/"/
/.=/! d
s/=/\#/' | awk -- '
BEGIN {
   FS="#";
   printf("<?xml version=\"1.0\"?>\n");
   printf("<authconfig>\n");
   printf("  <globals>\n");
   SETTING="false";
}
!/^ / {
   if (NR>1) {
   if (SETTING=="true")
      { printf("/>\n");
        printf("    </key>\n") }
   else
      { printf("/>\n") }
   }
   SETTING="false";
   printf("    <key name=\"%s\" value=%s",$1,$2)
}
/^ / {
   if (SETTING=="false")
      { printf(">\n");
        printf("      <settings")
   }
   printf("%s=%s",$1,$2);
   SETTING="true";
}
END {
   if (SETTING=="true")
      { printf("/>\n");
        printf("    </key>\n") }
   else
      { printf("/>\n") }
   printf("  </globals>\n");
   printf("</authconfig>\n");
}'

Now for my next trick....

In looking for my next challenge, I have decided to step away from the Oracle stuff for now and am going to try my hand at something down the stack.

I have used Openfiler for a while whenever I need a 'quick-and-dirty' SAN for any of my projects. It does the job I need it to do, but is getting a bit long in the tooth - the last official release was over 2 1/2 years ago. A significant problem is that it is based on the now-defunct rPath Linux distribution. About this time last year, Openfiler announced they would be moving back to a CentOS based distribution, releasing a preliminary download, but nothing has been heard since.

So I have decided to take up the baton.

Based on what I have seen so far, I am going to have to learn a lot more about the ins-and-outs of Linux storage, authentication, kernel building, rpm packaging and PHP than I ever thought I would, but maybe that is a positive thing.

My initial goal is to get the current Openfiler functionality on a vanilla CentOS 6 install, using only currently available repositories (i.e. CentOS, EPEL, ATrpms et al).  I want to avoid having to maintain my own versions of code that already exists. As an example I have found so far, Openfiler relies on getting an XML representation of the output of 'authconfig'. The current version of 'authconfig' does not produce this output. Given 'authconfig' is a Python script, it is reasonably easy to put the functionality back in, but then I would be left having to keep track of developments in 'authconfig' and maintaining my own branch.  In the tradition of UNIX, I would rather develop something that can take input from the standard 'authconfig' and output what Openfiler needs.

Once I get the basics of presenting the core storage services going well on stock CentOS code, then I might start looking into things like SCST and DBRD/clustering.

I think my biggest problem is going to be stopping from jumping all over the place and just focusing on one piece at a time :-)

Anyway, expect semi-regular updates on my progress. And help or advice is always welcome.