/home/jeff/blog

I was talking with someone this morning about ARMs and the sort of financial havoc they have been wreaking upon a population who was assured that they would be able to refinance before the “adjustable” portion of the mortgage kicked in. During that conversation, I realized the reason why banks were so interested in getting people into ARMs, and it didn’t really have anything to do with trying to weasel houses away from people.

Adjustable Rate Mortgages usually have a rate adjustment that kicks in after a few years. Quoting the fed: “With most ARMs, the interest rate and monthly payment change every month, quarter, year, 3 years, or 5 years. The period between rate changes is called the adjustment period. For example, a loan with an adjustment period of 1 year is called a 1-year ARM, and the interest rate and payment can change once every year; a loan with a 3-year adjustment period is called a 3-year ARM.”

These “financial products” have been sold with the idea that buyers would want to refinance with a traditional “fixed rate” mortgage before the adjustable period kicks in. For borrowers who would not otherwise qualify for a traditional mortgage, this gave a way to increase credit score through timely mortgage payments as well as the ability to “own” a house. (Not that any of this stuff is “owned” until the bank isn’t able to take it away for sometimes arbitrary reasons, but whatever…)

I contend that their intent is to keep people from owning (and I do mean owning as in “don’t owe the bank money”) their own house by minimizing the amount of potential equity in their houses. They can accomplish this by the way that mortgages work — at the beginning, virtually none of your monthly payment is applied to the principal (increasing equity), and at the end, almost all of it is applied to the principal. But — what if you never were able to get beyond a few years in a mortgage, as banks push for loans which have to be refinanced quickly to avoid financial penalties? This way, banks get your monthly payments and have to yield very little equity before the loan is renewed or sold to another vendor, at which point the bank receives the remainder of the money beyond the smidgen of equity they have yielded to the borrower. It’s almost like printing money. And we bailed these guys out, why?

In addition, my “financial advisor” pointed out that in order to refinance, if you obtain an new loan for the exact amount to payoff the existing mortgage you still have to suck out more equity because with closing costs, points, new mortgage title insurance, et cetera. You are going to end up borrowing an additional four to seven thousand dollars more just to get out of the old deal, not including any fees for pre-payment penalties. So, often times people are just stuck because they don’t have sufficient equity to refinance, or the cash reserve to refinance out of the deal (especially with the 80/20 deals that were so popular in the early 2000s).

I have been recently been trying to redesign FreeMED (my opensource GPL’d EMR/PM system) in order to work in a “clustered” environment, so that I could support scenarios where multiple application servers were load-balanced to handle larger quantities of traffic. The latest piece of this has been to move filesystem-based storage into the database layer so that I don’t have to mess around with clustered file storage and replication. Some of the highlights of this have been:

• Database-based session handling – At least when dealing with stateless applications like FreeMED, session information has a bad habit of being confined to the local filesystem. We ended up using PEAR’s HTTP_Session2 to support database-backed sessions. I can’t help but think this would be so much easier using JAAS …
• API abstraction – Don’t write this one off too quickly. A good API layer means that whenever you have to make these sweeping changes, you’re only playing with a few files of source code, as opposed to completely rewriting a good chunk of your codebase. Without the few layers of API which I had been using, I would not have been able to implement the database backed information store which I just completed.
• Database backed file store – This was the part I had just finished up. Storing files locally and relying on filesystem-based replication from some of the new clustering filesystems may *seem* like a good idea, but it’s not. Database gives you a single authoritative source, and unless we’re looking at multi-gigabyte files, there should not be an appreciable toll. Besides, you indexed your tables properly, right … ?
• Design for concurrency – That record locking may seem like a pain, but two people working on the same object can be catastrophic. Take the time to add a locking field if you have to.
• Singletons – Take “singleton” processes into account, so you’re only running a single iteration of something which can only be run once.

Those are the major bullet points, although I’m sure that upon review of this, I’ll think of some other design paradigms I had to play with to get this to function properly. Off to enjoy the weekend …

In 2007, I had posted a patch for the ADS DVD Xpress DX2 device to work on Linux, but it had been based on an antiquated kernel version, etc.

Since then, someone was nice enough to post an updated version of the driver, but without DVD Xpress DX2 support. I put together a patch which ensures that the drivers now compile and use the new I2C and V4L2 APIs. I can’t guarantee that it works, only that it compiles the driver properly now. Theoretically it should work, but I can’t find my DVD Xpress DX2 to try out the hardware properly.

Original driver : http://go7007.imploder.org/wp-content/uploads/2009/11/wis-go7007-linux-098-5b.tgz
Patch : http://jeff.ourexchange.net/wp-content/uploads/2010/03/wis-go7007-linux-098-5-20100325.diff

I do not like camera phones in the same way that I don’t particularly care for flash photography; you can produce good results (or at least passable results) with either, but the majority of stuff that comes out of it is just pure crap.

I recently moved to one of Sprint’s CDMA Android offerings, the HTC Hero. I just finished the flashing/rooting process, since I don’t particularly care for devices that try to lock me out of their own functionality. (The key to doing this with the “newer” Sprint CDMA Hero phones is apparently using an older version of the recovery image and making sure you have enough available memory to run the image flash, but that’s beside the point.) This particular device came with a 5 megapixel built-in camera, and though I know that resolution != quality, it does seem to produce *passable* images. Not great, but passable. I’m hoping to see an improvement when the CDMA Hero sees “official” Android 2.1 support. This OS version fragmentation is bunk, but at least I can upgrade as soon as the pieces are available for another model. Suck that, closed source technology.

Am I giving up the 40D? Hell emphatically *no*. But I can capture a few things I might miss fumbling for the camera bag. At least, I hope I can.

I have just wrapped up (no pun intended) work on an initial version of a CLI JasperReports wrapper, based heavily off the work of jasperCall. It’s also quite similar to the work being done on RunJasperReports, although it was specifically designed to be integrated into FreeMED’s reporting engine, as it is put together as a fatjar. It currently supports PDF, XML, XLS and HTML output, and should theoretically support parameter passing, though I haven’t tried it out yet. It has a newer version of MySQL Connector/J baked in as well, so it shouldn’t require any external libraries outside a standard JRE install to function properly.

The code can be check out of anonymous Subversion from https://svn.master.freemedsoftware.org/freemed-utilities/org.freemedsoftware.util.JasperWrapper/ if anyone is interested in building it. (It obviously should require Java 6 JDK to compile properly.) Hopefully I’ll get around to tagging and releasing versions of it for download shortly, if anyone shows interest in it.

I’ve decided to do a little feature on films that I liked for one reason or another, and to start out, I have chosen a scifi/comedy flick from the 1980s called The Adventures of Buckaroo Banzai Across the Eighth Dimension.

It’s a pretty funny send-up of scifi movies which take themselves too seriously. I think that after watching it, it’s pretty apparent that none of the actors were told that it was supposed to be a comedy. Peter Weller plays the entire movie completely straight, which makes his lines all the funnier. John Lithgow was very funny, in his strange and quirky way.

It was apparently very difficult for the studio that owned it to release “Buckaroo Banzai” because of some legal issues, but it finally made it to DVD in the last few years.

• iMDB Link
• Netflix Link

The simple recipe is to add this to root’s cron:


* * * * * /usr/bin/gmetric -c /etc/gmond.conf -n pf_states -v $(/usr/local/sbin/pftop -b | grep pfTop | cut -d/ -f2 | cut -d, -f1) -t int32 -d 120 2>&1 | logger -t pf_states


and install the pftop package along with a gmetric binary and a working /etc/gmond.conf configuration file. It might be advantageous to check for the maximum number of states as well.

In addition, you might want to know which pf rules are passing how much traffic. A nice easy way of doing this is to create this file as ./pfstates (make it executable, of course):

#!/usr/bin/perl
# pfstates
# jeff@ourexchange.net
my $limit = shift  || 0;

my $seg = 0;
my @s = [];

while (chomp( my $line = <>)) {
        $s[$seg] = $line;
        if ($seg == 2) {
                $seg = 0;
                if ($s[1] !~ /States\: 0/) {
                        my $states = 0;
                        if ($s[1] =~ m/States\: (\d+)/) {
                                $states = $1;
                        }
                        if ($states >= $limit) {
                                print "[$states] $s[0]\n";
                        }
                }
        } else {
                $seg++;
        }
}

…. then you would pipe pfctl’s state output to it:

pfctl -v -s rules | ./pfstates

Optionally you could add a “minimum level” of connections you want to see:

pfctl -v -s rules | ./pfstates 100

for example to see only rules passing 100 or more active connections.

I have a hate/hate relationship with Exchange Server. I hate it, and I’m pretty sure it hates me.

Why someone would design a system to expose every bit of data for a system through a nice standard protocol like IMAP, then only allow certain things to be viewed through a piece of crap proprietary protocol like MAPI just boggles the mind. I’m sure it’s part of their “vendor lock-in” thing, but it just pisses me off.

Anyway, I found out that to completely backup an Exchange account (as I have been doing over the last week or so for different accounts), you also have to backup the non-mail portions. I ended up using a deprecated library called Jakarta Slide for WebDAV client support, which helpfully came with a SearchMethod call which was capable of running the specialized XML query required to backup the data.

The XML I ended up using was:

<?xml version="1.0"?>
<a:searchrequest xmlns:a="DAV:">
<a:sql>
SELECT * FROM "URL"
</a:sql>
</a:searchrequest>

in case anyone is interested. Again, I have a fatjar of this utility, but I have to check with work to make sure they’re okay with me releasing it before I can post it anywhere.

I hate it. IMAP Synchronization, that is.

In an effort to migrate users from one *shudder* Exchange provider to another (after getting shot down for proposing first Zimbra, then standard mail server stuff, then Openchange), I have been going through all of the available IMAP sync software that I could find.

mbsync (http://isync.sourceforge.net/mbsync.html) – We use this for IMAP backup, so I figured it would be a good idea to try it for syncing between two IMAP servers. *Crash*. *Segfault*. This is one of the reasons I’m not a big fan of really low-level apps written in C.

imapsync (http://www.linux-france.org/prj/imapsync/) – Written using a Perl library, this offered the promise of fast syncing, and appeared to work in test runs. That is, until it encountered folders with spaces in them, and threw up all over the terminal. Really, it wasn’t a use-case for *anyone* to have spaces in folder names? I could have put some effort into fixing the library and/or script, but I figured that if something that basic wasn’t working for a fairly mature library, this wasn’t going to be pretty.

The Imap Migration Tool (http://migrationtool.sourceforge.net/) – Written in PHP, with a two or three step process. Was meant for moving stuff, not keeping them in sync, so running multiple times would cause additional copies of messages to be created. Also, it seems to have been written in some prehistoric form of PHP, which required hacking to get it even marginally functional. Even then, it spat tons of warnings and nasties, which made me a little nervous to use it on any important mailboxes.

mailsync (http://mailsync.sourceforge.net/) – I don’t even remember why this one didn’t work, only that I wasted a fair amount of time playing with it to get it to try to migrate using the scheme we were working with to no avail. Scratch another one.

Solution … I had to roll my own. This is *stupid* to the highest degree, but I rolled something using javamail into a self-contained “fat jar” using fatjar. It works sickeningly well, and actually respects existing messages by message id, so that if for any reason I have to break the sync process, it’ll skip past anything it has already done. I made it non-destructive because it seemed like seriously bad juju to have a sync tool purge out email…. If I get permission from work, I’ll post the jar for anyone with Java 1.6 to use.

As an aside, it took a little bit of hackery to get fatjar running outside eclipse. The relevant section of build.xml was:

  <path id="fatJarPath" location="lib/fatjar.jar"/>
  <property name="ant.reuse.loader" value="true"/>

  <taskdef name="fatjar.build" classname="net.sf.fjep.anttask.FJBuildTask" classpathref="fatJarPath"/>
  <typedef name="fatjar.manifest" classname="net.sf.fjep.anttask.FJManifestType" classpathref="fatJarPath"/>
  <typedef name="fatjar.exclude" classname="net.sf.fjep.anttask.FJExcludeType" classpathref="fatJarPath"/>
  <typedef name="fatjar.jarsource" classname="net.sf.fjep.anttask.FJJarSourceType" classpathref="fatJarPath"/>
  <typedef name="fatjar.filesource" classname="net.sf.fjep.anttask.FJFileSourceType" classpathref="fatJarPath"/>

  <target name="main" depends="build">
    <fatjar.build output="ImapSynchronizer.jar">
      <fatjar.manifest mainclass="...myclasspath...ImapSynchronizer"/>
      <fatjar.filesource path="bin" relpath=""/>
      <fatjar.jarsource file="lib/mail-1.4.3.jar" relpath=""/>
    </fatjar.build>
  </target>

I have recently had cause to proxy a J2EE CXF service through an apache 2.2 instance, and thought it would be nice to share my findings. (This was all done on a Debian system.)

First of all, the mod_proxy pieces have to be enabled using a2enmod proxy.

A fragment has to be added with the proxying bits and some limitation:

<Proxy proxy:http://SERVER:PORT/URL>
     Order allow,deny
     Allow from all
</Proxy>

ProxyPass /EXPOSEDURL http://SERVER:PORT/URL
ProxyPassReverse /EXPOSEDURL http://SERVER:PORT/URL

Reloading apache configuration should enable the proxy properly. The only other possible issue is that in addition to the WSDL URL, you’re going to have to specify a “proxy” URL, which is just the service URL without ?wsdl at the end of it. A fragment of PHP’s SoapClient would look like this:

$url = "http://server.domain:8180/services/MySoapService?wsdl";
$sc = new SoapClient( file_get_contents($url), array(
          'login' => $my_username // optional for basic auth
        , 'password' => $my_password // optional for basic auth
        , 'compression' => SOAP_COMPRESSION_ACCEPT | SOAP_COMPRESSION_GZIP
        , 'location' => str_replace('?wsdl', '', $url) // force this to work through proxies
));

The magic in that last segment was the “location” parameter, as it specifies the proxy. Good luck!