/home/jeff/blog

Archive for the ‘Hacks’ Category

The simple recipe is to add this to root’s cron:


* * * * * /usr/bin/gmetric -c /etc/gmond.conf -n pf_states -v $(/usr/local/sbin/pftop -b | grep pfTop | cut -d/ -f2 | cut -d, -f1) -t int32 -d 120 2>&1 | logger -t pf_states


and install the pftop package along with a gmetric binary and a working /etc/gmond.conf configuration file. It might be advantageous to check for the maximum number of states as well.

In addition, you might want to know which pf rules are passing how much traffic. A nice easy way of doing this is to create this file as ./pfstates (make it executable, of course):

#!/usr/bin/perl
# pfstates
# jeff@ourexchange.net
my $limit = shift  || 0;

my $seg = 0;
my @s = [];

while (chomp( my $line = <>)) {
        $s[$seg] = $line;
        if ($seg == 2) {
                $seg = 0;
                if ($s[1] !~ /States\: 0/) {
                        my $states = 0;
                        if ($s[1] =~ m/States\: (\d+)/) {
                                $states = $1;
                        }
                        if ($states >= $limit) {
                                print "[$states] $s[0]\n";
                        }
                }
        } else {
                $seg++;
        }
}

…. then you would pipe pfctl’s state output to it:

pfctl -v -s rules | ./pfstates

Optionally you could add a “minimum level” of connections you want to see:

pfctl -v -s rules | ./pfstates 100

for example to see only rules passing 100 or more active connections.

I have a hate/hate relationship with Exchange Server. I hate it, and I’m pretty sure it hates me.

Why someone would design a system to expose every bit of data for a system through a nice standard protocol like IMAP, then only allow certain things to be viewed through a piece of crap proprietary protocol like MAPI just boggles the mind. I’m sure it’s part of their “vendor lock-in” thing, but it just pisses me off.

Anyway, I found out that to completely backup an Exchange account (as I have been doing over the last week or so for different accounts), you also have to backup the non-mail portions. I ended up using a deprecated library called Jakarta Slide for WebDAV client support, which helpfully came with a SearchMethod call which was capable of running the specialized XML query required to backup the data.

The XML I ended up using was:

<?xml version="1.0"?>
<a:searchrequest xmlns:a="DAV:">
<a:sql>
SELECT * FROM "URL"
</a:sql>
</a:searchrequest>

in case anyone is interested. Again, I have a fatjar of this utility, but I have to check with work to make sure they’re okay with me releasing it before I can post it anywhere.

I hate it. IMAP Synchronization, that is.

In an effort to migrate users from one *shudder* Exchange provider to another (after getting shot down for proposing first Zimbra, then standard mail server stuff, then Openchange), I have been going through all of the available IMAP sync software that I could find.

mbsync (http://isync.sourceforge.net/mbsync.html) – We use this for IMAP backup, so I figured it would be a good idea to try it for syncing between two IMAP servers. *Crash*. *Segfault*. This is one of the reasons I’m not a big fan of really low-level apps written in C.

imapsync (http://www.linux-france.org/prj/imapsync/) – Written using a Perl library, this offered the promise of fast syncing, and appeared to work in test runs. That is, until it encountered folders with spaces in them, and threw up all over the terminal. Really, it wasn’t a use-case for *anyone* to have spaces in folder names? I could have put some effort into fixing the library and/or script, but I figured that if something that basic wasn’t working for a fairly mature library, this wasn’t going to be pretty.

The Imap Migration Tool (http://migrationtool.sourceforge.net/) – Written in PHP, with a two or three step process. Was meant for moving stuff, not keeping them in sync, so running multiple times would cause additional copies of messages to be created. Also, it seems to have been written in some prehistoric form of PHP, which required hacking to get it even marginally functional. Even then, it spat tons of warnings and nasties, which made me a little nervous to use it on any important mailboxes.

mailsync (http://mailsync.sourceforge.net/) – I don’t even remember why this one didn’t work, only that I wasted a fair amount of time playing with it to get it to try to migrate using the scheme we were working with to no avail. Scratch another one.

Solution … I had to roll my own. This is *stupid* to the highest degree, but I rolled something using javamail into a self-contained “fat jar” using fatjar. It works sickeningly well, and actually respects existing messages by message id, so that if for any reason I have to break the sync process, it’ll skip past anything it has already done. I made it non-destructive because it seemed like seriously bad juju to have a sync tool purge out email…. If I get permission from work, I’ll post the jar for anyone with Java 1.6 to use.

As an aside, it took a little bit of hackery to get fatjar running outside eclipse. The relevant section of build.xml was:

  <path id="fatJarPath" location="lib/fatjar.jar"/>
  <property name="ant.reuse.loader" value="true"/>

  <taskdef name="fatjar.build" classname="net.sf.fjep.anttask.FJBuildTask" classpathref="fatJarPath"/>
  <typedef name="fatjar.manifest" classname="net.sf.fjep.anttask.FJManifestType" classpathref="fatJarPath"/>
  <typedef name="fatjar.exclude" classname="net.sf.fjep.anttask.FJExcludeType" classpathref="fatJarPath"/>
  <typedef name="fatjar.jarsource" classname="net.sf.fjep.anttask.FJJarSourceType" classpathref="fatJarPath"/>
  <typedef name="fatjar.filesource" classname="net.sf.fjep.anttask.FJFileSourceType" classpathref="fatJarPath"/>

  <target name="main" depends="build">
    <fatjar.build output="ImapSynchronizer.jar">
      <fatjar.manifest mainclass="...myclasspath...ImapSynchronizer"/>
      <fatjar.filesource path="bin" relpath=""/>
      <fatjar.jarsource file="lib/mail-1.4.3.jar" relpath=""/>
    </fatjar.build>
  </target>

In case anyone may want to use Trixbox with Cisco 79xx phones and wants to use a directory with it, I have put together a hack to deal with the directory services, which nominally require SugarCRM, so that they report simple extensions back in the appropriate format.

1) In the Endpoint manager, make sure that the services URL and directory URL are in the default configuration for Cisco phones, and restart any phones which are running to activate the configuration.

2) In /var/www/html/cisco/services, make sure that index_cisco.php is symlinked to index.php

3) In /var/www/html/cisco/services/DBConnect.inc, change the default username and password to something with access to the asterisk database, and change the database name to asterisk.

4) Make PhoneDirectory.php in that directory contain:

< ?php

##########################################
# Config section

$Server = "http://$SERVER_ADDR/cisco/services";   # don't add a trailing slash!
$LongDistanceExtension = ""; # change this to match your config

##########################################
#
# Set content type
header("Content-Type: text/xml");

# This is the include file that establishes the connection to the MySQL database
require ("DBConnect.inc"); 

# This sets the offset for the LIMIT portion of the query
$NextStartingRow = $NextSet*30;

# Connect to the database
$ConnectionSuccess = db_connect();
if (!$ConnectionSuccess) exit;

# If the variable "ID" is passed in through the GET string, then display
# extension, phone number and cell phone number for that record with the dial
# key functionality
if ($ID) {
        $PersonDirectoryListing = "\n";

        //$Query  = "SELECT id, first_name, last_name, phone_home, phone_work, phone_mobile, phone_other ";
        //$Query .= "FROM contacts WHERE id = '$ID' ";
        //$Query .= "ORDER BY last_name ";
        $Query  = "SELECT id,description FROM devices WHERE ( tech = 'sip' OR tech = 'iax' ) AND id = '".addslashes($ID)."' ";

        $SelectPersonInfo = mysql_query($Query,$ConnectionSuccess);

        while ($row = mysql_fetch_array($SelectPersonInfo)) {
                $WorkPhone = ereg_replace("[ ()-]+", "", $row['id']);

                $PersonDirectoryListing .= "\n";
                $PersonDirectoryListing .= "Extension:\n";
                //$PersonDirectoryListing .= "$LongDistanceExtension$WorkPhone\n";
                $PersonDirectoryListing .= "$WorkPhone\n";
                $PersonDirectoryListing .= "\n";
        }

        $PersonDirectoryListing .= "\n";

        echo "$PersonDirectoryListing";

# If the variable ID is not passed in on the GET string, then do the
# entire company directory, unless last is passed in. If so then we
# will be using a LIKE filter in our SQL query
} else {
        $PersonNameList = "\n";
        $PersonNameList .= "\n";
        $PersonNameList .= "
Please select one\n";

        $Query  = "SELECT id,description FROM devices WHERE ( tech = 'sip' OR tech = 'iax' ) ";
        //$Query  = "SELECT id, first_name, last_name, phone_home, phone_work, phone_mobile, phone_other ";
        //$Query .= "FROM contacts WHERE deleted = 0 ";

        # If we are searching by last name the add this filter to the query
        if ($LastName) { $Query .= "and description like '%$LastName%' ";  }
        $Query .= "ORDER BY id ASC";

        # If this is the first page of the company directory then we will display the first 30
        if (!$NextSet) {
                $Query .= " Limit 0,30";
        # Now for each subsiquent call we get the next 30 records.
        } else {
                $Query .= " Limit $NextStartingRow,30";
        }

        # Execute the query
        //echo $Query;
        $SelectNameList = mysql_query($Query,$ConnectionSuccess);

        # Count the number of rows returned. This is important because if a full 30 are returned
        # we will display a more option

        $NumberOfRows = mysql_num_rows($SelectNameList);

        if ($NumberOfRows >= 30) {
                $NextSetValue = $NextSet+1;
        }

        # Parse through the query and set up the menu items.
        while ($row = mysql_fetch_array($SelectNameList)) {
                $PersonNameList .= "\n";
                        $PersonNameList .= "";
                                $PersonNameList .= $row["description"];
                                if ($row["first_name"]) $PersonNameList .= ", " . $row["first_name"];
                        $PersonNameList .= "\n";
                        $PersonNameList .= "";
                                $PersonNameList .= "$Server/PhoneDirectory.php?";
                                $PersonNameList .= "ID=";
                                $PersonNameList .= $row["id"];
                                $PersonNameList .= "\n";
                $PersonNameList .= "\n";
        }

        # If we set NextSetValue above then we will display the more option. Which sets NextSet
        if ($NextSetValue) {
                $PersonNameList .= "\n";
                        $PersonNameList .= "MORE\n";
                        $PersonNameList .= "$Server/PhoneDirectory.php?NextSet=$NextSetValue\n";
                $PersonNameList .= "\n";
        }
        $PersonNameList .= "";
        echo "$PersonNameList";
}
?>

The only haproxy load balancer package for the NSLU2 is really, really old, and is installed as “optware” in /opt. With that in mind, here’s the 1.3.15.3 package, installed in root.

Package : haproxy_1.3.15.5-2_armv5b.ipk
Source : haproxy-1.3.15.5.tar.gz

Instead of having to deal with an entire Perl installation on an NSLU2, I compiled a readily available urlencode binary, which takes piped input and encodes it for use in a URL.

Tarball : urlencode-armv5b.tar.gz

For all those fans of the venerable NSLU2 (or “slug” as we like to call it), I have another package for OpenSlug/BE. This time it’s remserial, a Linux equivalent to the BSD “netfwd” software, allowing serial ports to be redirected over TCP.

Package: remserial_0.2000-1_armv5b.ipk
Source: remserial.tar.gz

For those enterprising people who would like to use my armv5b-softfloat-linux cross compilation toolchain, I have made it available on mediafire as well. It was compiled with Ubuntu 8.04.1 i386, so that’s the safest bet to use for compilation. I had used the NSLU2 “master makefile” to build it, since the vanilla crosstool had provided me little in terms of usable toolchains for the NSLU2.

Before a friend clued me in to the embedded gmetric project, I had needed to strip out just the gmetric binary for use in monitoring a process on the NSLU2 I was using.

So, for anyone who is interested, I have posted both a binary ipkg package for OpenSLUG/BE and a source package containing all of the pieces I yanked from the ganglia 3.1.1 distribution to create it. (Graciously hosted by mediafire.com…) Hopefully this can be of use to someone.

This past week, I stumbled on a strange problem with Dell “PERC” RAID controllers in certain rackmount servers, where they would suddenly just stop working for no apparent reason. The nice people at Dell did actually have firmware available for the PERC 3 and 4 controllers to fix this problem, which was pretty nice.

The part that *wasn’t* nice was that they were some stupid Windows binaries which required a floppy disk to be inserted.

First of all, who has a laptop with a floppy drive anymore, and second of all, what if you, like other sane people, don’t use Windows? Thirdly, how often does a floppy disk magically fail to work in a floppy drive?

I ended up doing the following:

• Create a VirtualBox VM
• Use dd if=/dev/zero of=floppy.img bs=1k count=1440 to create a floppy image, which I mounted in the VirtualBox
• “Format” the floppy image in the VM, because apparently Dell’s fuqtard utility can’t write an image to an unformatted disk.
• Use the stupid utility to write the floppy image.
• Use mkisofs -pad -b floppy.img -R -o cd.iso floppy.img to create a CD image

Dell, I hate you. Your laptops overheat and suck out loud, and your stupid Windows-only disks and defective RAID controllers bother me. I’m not buying any more of your hardware. You can subsidize Microsoft some other way.

I finally got around to getting a 12VDC power supply for the car (15 USD, cheap!), so my WRT54G3G-ST is mobile, and I got to enjoy trying out Skype while doing 65mph on the Mass Pike.

There’s some sort of gee-whiz factor in all this, but I guess it’s just another way to become more and more endlessly connected.