MarcEdit Swap Field function updates

I meant to post this earlier, but then got busy with travel.  Three weeks ago, I was sitting down to make a few changes to the swap field function in MarcEdit and realized after scanning the function, I couldn’t remember how it worked.  This doesn’t happen very often (I probably over document my code), but I hate when it does, and when it does, it’s time to refractor (something I really try not to do very often).  The problem is the swap field function in general.  I’ve known for sometime that I was going to have to address it.  In fact, in the comments header of the function, I had written two years ago (when I first started to think about doing this)

‘****************************************************************************************
* For the love of God, don’t work on this function.  You’ll only break it.
* trust me! — TR 2011-10-17
****************************************************************************************’

Apparently, I must have been interested in enhancing the function back in 2011, and came to the same conclusion I’d come to this past month.

The problem with the Swap Field function is that it’s old…older than old.  If MarcEdit was the Tardis, the swap field function would be like the 3rd Dr. old.  It’s been around forever, and was part of the original codebase written in Assembly.  And that was my problem.  When MarcEdit migrated from Assembly to C#, I rewrote nearly the entire program — you pretty much had to.  But there were a few functions that were really complicated, and they worked, so rather than rewrite, I ported….and porting code from Assembly to a higher level language can sometimes have ugly results.  In my case, 12K lines of ugly.  So, I’ve been ignoring, until I couldn’t any longer.

So what was the enhancement I couldn’t ignore?  Well, it was a request for the tool to do something that it probably ought to, but I hadn’t considered.  And when I looked at the function, I realized that it would take me longer to figure out how to bolt it onto the existing function than it would to rewrite from scratch…and at that point, I started planning the rewrite.

When rewriting a significant amount of code like this, it’s a great opportunity to redefine the scope of the tool, and that’s what I did here.  I had one option in the swap field tool called “Override Default behavior”.  At some point, that meant something to me — but no longer.  I decided to drop it.  It also gave me a change to include new functionality like joining subfields together, creating new fields conditionally, adding support for multiple field appending, etc.  I was able to plan my development so that I could create a new framework for the functionality that would allow me to close 10 or so enhancement requests and provide flexibility to grow into the future.  And that’s what I did.

The total rewrite process took approximately 10 hours (a weekend), reduced the total code for the function from 12k down to around 2.5-3K, and become much more modular, and I was able to break it up into smaller code parts and reuse some already existing functions that exist to support other MarcEdit functions.  Overall, I’m very happy with the result.

In the end, the Swap Field function window did change too:

From the above, you can see the new find textbox in the Modified data area, as well as the add to existing/create new option.  The program now also supports joining multiple subfields together — so for example swapping subfields ab from a 050 into a 500 a can be done by setting the Original Field Data subfields to ab and the Modified Data Subfields to a — the program will join the resulting subfield data together.  Lots of small enhancements, as well as improved performance.  All good things.

Of course, the best part of rewriting this function is that I understand it again, and it’s now much easier to understand because the code guides you through the process.  And I got to test this out last week.  I released the updated code 2 weeks ago, and in the first week, a couple of bugs were reported, and correcting those bugs ended up being a trivial exercise.  I also got an enhancement request, and that could easily be accommodated thanks to the new scaffolding.

Anyway, I bring this up for two reasons.  One, I see the first month or so as a shake out period, so if you are a MarcEdit user and happen to see a problem with the function, or you notice the behavior has changed and not for the better, let me know.  I’ve found in the last week that people made heavy use of some undocumented behavior (and by undocumented, I mean, completely unintentional), and as long as it makes sense, I’ll formalize the support.

The second reason is that I think a lot of times, we can let things go and stop innovating because things work and we don’t necessarily have the tools to move it forward.  You may not print it out as plainly as I did in the source code — but it definitely happens and when it does, services stagnate and we get, maybe, a little too attached to the status quo.  Making changes in these circumstances is hard, its painful, and it sometimes means taking a step backward…but in the end, they can provide a much clearer path to future innovation.

–tr