OpenFiler 2.99.1 on DL385G3

Description

This page describes how to configure OpenFiler version 2.99.1 on bare metal, and work through some of the issues.  I had tested version 2.3 some years ago, but settled on FreeNas7 for my home network shares.  Now I want to use a spare HP DL385G3 server, and create an iSCSI SAN, so it was between FreeNAS8 and OpenFiler 2.99.1.  It took me about 6 different installation attempts before I got it right, what with the updates screwing me up, the LDAP store getting corrupted, the USB drives getting lost on reboot, lost shares at reboot, and other such issues.  But in the end, I liked OpenFiler better, and was able to work out my issues with it. 

Process/Implementation

As always, it begins with a download.  But unlike the virtual appliance method for VMWare, this takes a little more effort.  So here are the steps I followed to get my setup working the way I wanted :
 

• Download the OpenFiler 2.99.1 x64 release. 
• Burn the ISO to a cdrom, I tend to use imgburn for that.
• Disconnect any of the external drives at this point, because during the installation there will be some confusion about where the bootblock will go, and even though you tell it what you want, it seems to get screwed up.  So trust me, leave the external drives unplugged for now.
• Insert the cdrom and boot the server.
• When the installation comes up - choose text mode.  Again, trust me here - because I had too many issues with installations hanging on me, or causing issue - when using the graphical installation.  Using the text mode seemed to resolve all those issues.
• There are some good pages on how to answer the installation questions (here and herefor example), so I'm not going to get into all the details.  But just note that I chose to only clear partitions and setup the primary drive.  I left all other drives alone.  I say all other drives, because I had all 6 bays populated in my configuration - 2 x 36Gb RAID-1 for the O/S image, and 4 x 36Gb RAID-5 for testing and shares.  Its the 4 x 36's I'm talking about leaving alone here ... and of course my USB drives were already disconnected.
• After following all the on screen questions, reboot.
• The system is now up and ready for configuration!

Tuning / Customization

From that point, its all about tuning/customization to get it to "fit" into my infrastructure.  Configuration for me consisted of getting the additional NIC's configured, adding my external storage, configuring my volume groups, volumes, shares, and access.  Seems simple enough, but it took quite a few trials to get right.  Here are the steps used in the last (successful) installation.

• Login to the OpenFiler gui at https://openfiler.yourdomain.com:446/, default user is "openfiler" and default password is "password".
• First thing to do is to reset the admin password.  So go to the "Accounts" tab, and then hit the "Admin Password" link in the "Action Block" which is in the upper right area.
• Click on the "System" tab, and then "Network Setup" from the Action Block
• Click on the "Accounts" tab, and then configure LDAP (need exact info here)
• While still in the Accounts tab, click on "Administration" from the Action Block to confirm LDAP works .. it will error out if not.
• Click on the "Services" tab, and enable CIFS, NFS, LDAP (if not already on), iSCSI Target, and leave the rest off - at least I did
• While still in the "Services" tab, click on the "iSCSI Target Setup" link in the Action block to create an incoming CHAP user and password (which you'll need to type in within vSphere discovery), and enable the iSNS Server, leveraging the server's own IP address for that.

• I wanted a raid array of my three USB external drives.  So I went into the "Volumes" tab, and on the right side, choose "Add Volume".

• The key here is to make sure that when adding a device, choose the type of "RAID Array Member" - that's the key.  I had to do this for all three drives.  I basically carved up my drives into one big partition, which I made the RAID slice.  This means that this is now basically a physical volume, which we can carve out into smaller parts using LVM.

• To make the first usable logical volume, go into the "Volume" tab again, and construct a volume from those raid slices.  I had 2TB drives, so I carved up a 1.5TB volume, another 1.5TB volume, and left the rest free for now.  Note that when carving up a volume, it wants the file system type.  Resist the temptation of using ZFS/BTRFS - because at least for me, this created such problems for machine that it wouldn't stay up past 20 hours.  So I recreated as EXT4 and its been rock solid (and faster) ever since.

• Run the System Update from the upper right hand mini menu, and update these packages :
  • hdparm
  • kernel*
  • mdadm
  • openldap*
  • php-openfiler
  • sysfsutils
  • udev
  • userspace-kernel*
  • openfiler*
  • lvm2
  • nfs-client
  • nfs-server - which causes NFS / mountd to stop working
  • samba
  • zfs
  • group-openfiler
• We need to fix broken NFS/mountd now, so we will follow info here : http://forums.openfiler.com/viewtopic.php?pid=26145#p26145)
  • service nfslock stop
  • service nfs stop
  • service portmap stop
  • service rpcbind stop
  • chkconfig rpcbind on
  • chkconfig nfs on
  • chkconfig nfslock on
  • chkconfig portmap off
  • service rpcbind start
  • service nfs start
  • service nfslock start
• Edit the /etc/init.d/nfs script, line 185 area in the "status" stanza, change the typo of running "status nfsd" to "status nfs".  The extra "d" is what's killing the status routine, and causing the Servicse page in OpenFiler to report NFS as down.
• Edit the "/opt/openfiler/var/www/includes/generate.inc" and :
  • add a new line at line number 1789 to set the workgroup to my own personal workgroup.  This is what will override the auto generation stuff that wipes out manual settings in smb.conf.
  • edit line 877 to change chmod from 2770 to 2775
  • edit line 1152 to change create mode from 0770 to 0775
  • edit line 1153 to change directory mode from 2770 to 2775
  • add a new line at line number 1811 to set "log level = 2".  This helps troubleshoot why the atron keeps getting access denied.
  • edit line 1853 to remove the semicolon/comment character and enable the use of the smbusers file
• For things like the audiotron to work, I was unable to use LDAP - it caused too many issues with NTLM authentication.  Samba logs were reporting everything from no user, to bad password, so everything in between.  Using "Security = server" mode was the ticket, and this was achieved by turning off LDAP, and enabling NT4 authentication.  Also - very key was setting "no default domain" in the SMB/CIFS settings because that creates the "Security = server" mode - otherwise it goes to "security = domain".
• Should be good to go now.
  

Then just reboot the system, but ONLY after confirming that the background tasks are completed.

Further Information

There are some good sites out there with information about this stuff.  One such site is about configuring VMWare vSphere for iSCSI.  One of the tough things to find is actual documentation - well, tough to find for free.  There is still the old version of the old OpenFiler documentation which is still helpful, albeit a little outdated.  I actually preferred to work with the PDF version, because it was easier to print and also easiser to search through.  It covers the older 1.1 version, so the screens are a little dated, but for the most part, it covers the basics.  For example, page 5 shows how to login - which I didnt know by default.  Same for configuring services, adding volumes, etc.

The script I setup to get around the issues with software raid is shown below. I put it in /etc/init.d/software-raid-fix.

#!/bin/sh
#
# chkconfig: 345 99 01
# description: # Fix software raid not auto-assembling
#
# ident "@(#)software-raid-fix 2.0 - (C) pluzzi Software 2008"
##############################################################################
#
#
#####
#####  Added by Paul A. Luzzi on 11/04/2011
#####
#####  Purpose : This section added to fix software raid not auto-assembling.
#####    Its designed to find missing volume groups, activate then, search
#####    for any filesystems in those volume group volumes, and reshare them.
#####
#####  Tested on OpenFiler 2.99.1 ... not sure if this works on OF 2.3
#####

# Source function library.
. /etc/rc.d/init.d/functions

# See how we were called.
case "$1" in
  start)
        #
	echo "Collecting existing VGs : "
        vgdisplay | grep "VG Name" | awk '{print $NF}' > /tmp/vgd.boot
        #
	echo -n "Assembling the meta devices : "
        /sbin/mdadm --assemble --scan
        #
	echo "Collecting current VGs : "
        vgdisplay | grep "VG Name" | awk '{print $NF}' > /tmp/vgd.post
        #
        echo "Activating the volume group for meta based groups : "
        diff /tmp/vgd.boot /tmp/vgd.post | grep "^>" | awk '{print $NF}' | while read VGNAME
          do 
            echo -n "Activating volume group $VGNAME "
            /sbin/vgchange -ay $VGNAME
          done
        #
        echo "Mounting filesystem shares :"
        ## mount -a
        lvdisplay | grep /dev | awk '{print $NF}' | while read DEVICE
          do
            echo -n "Mounting $DEVICE at $MOUNTPOINT"
            FSTYPE=`grep -w "$DEVICE" /etc/fstab | awk '{print $3}'`
            MOUNTPOINT=`grep -w "$DEVICE" /etc/fstab | awk '{print $2}'`
            DEVICENAME=`grep -w "$DEVICE" /etc/fstab | awk '{print $1}'`
            mount -t $FSTYPE $DEVICENAME $MOUNTPOINT
          done
        #
        echo -n "Restarting NFS to catch any new shares :"
        /etc/init.d/nfs restart
        #
        echo -n "Restarting SMB to catch any new shares :"
        /etc/init.d/smb restart
        #
	echo
	;;
  stop)
	echo -n "No action needed for $0"
	echo
	;;
  *)
	echo "Usage: $0 {start|stop}"
	exit 1
esac

exit 0

Thats it!