May 302013
 

I meant to post this earlier, but then got busy with travel.  Three weeks ago, I was sitting down to make a few changes to the swap field function in MarcEdit and realized after scanning the function, I couldn’t remember how it worked.  This doesn’t happen very often (I probably over document my code), but I hate when it does, and when it does, it’s time to refractor (something I really try not to do very often).  The problem is the swap field function in general.  I’ve known for sometime that I was going to have to address it.  In fact, in the comments header of the function, I had written two years ago (when I first started to think about doing this)

‘****************************************************************************************
* For the love of God, don’t work on this function.  You’ll only break it.
* trust me! – TR 2011-10-17
****************************************************************************************’

Apparently, I must have been interested in enhancing the function back in 2011, and came to the same conclusion I’d come to this past month.

The problem with the Swap Field function is that it’s old…older than old.  If MarcEdit was the Tardis, the swap field function would be like the 3rd Dr. old.  It’s been around forever, and was part of the original codebase written in Assembly.  And that was my problem.  When MarcEdit migrated from Assembly to C#, I rewrote nearly the entire program – you pretty much had to.  But there were a few functions that were really complicated, and they worked, so rather than rewrite, I ported….and porting code from Assembly to a higher level language can sometimes have ugly results.  In my case, 12K lines of ugly.  So, I’ve been ignoring, until I couldn’t any longer.

So what was the enhancement I couldn’t ignore?  Well, it was a request for the tool to do something that it probably ought to, but I hadn’t considered.  And when I looked at the function, I realized that it would take me longer to figure out how to bolt it onto the existing function than it would to rewrite from scratch…and at that point, I started planning the rewrite.

When rewriting a significant amount of code like this, it’s a great opportunity to redefine the scope of the tool, and that’s what I did here.  I had one option in the swap field tool called “Override Default behavior”.  At some point, that meant something to me – but no longer.  I decided to drop it.  It also gave me a change to include new functionality like joining subfields together, creating new fields conditionally, adding support for multiple field appending, etc.  I was able to plan my development so that I could create a new framework for the functionality that would allow me to close 10 or so enhancement requests and provide flexibility to grow into the future.  And that’s what I did.

The total rewrite process took approximately 10 hours (a weekend), reduced the total code for the function from 12k down to around 2.5-3K, and become much more modular, and I was able to break it up into smaller code parts and reuse some already existing functions that exist to support other MarcEdit functions.  Overall, I’m very happy with the result.

In the end, the Swap Field function window did change too:

image

From the above, you can see the new find textbox in the Modified data area, as well as the add to existing/create new option.  The program now also supports joining multiple subfields together – so for example swapping subfields ab from a 050 into a 500 a can be done by setting the Original Field Data subfields to ab and the Modified Data Subfields to a – the program will join the resulting subfield data together.  Lots of small enhancements, as well as improved performance.  All good things.

Of course, the best part of rewriting this function is that I understand it again, and it’s now much easier to understand because the code guides you through the process.  And I got to test this out last week.  I released the updated code 2 weeks ago, and in the first week, a couple of bugs were reported, and correcting those bugs ended up being a trivial exercise.  I also got an enhancement request, and that could easily be accommodated thanks to the new scaffolding.

Anyway, I bring this up for two reasons.  One, I see the first month or so as a shake out period, so if you are a MarcEdit user and happen to see a problem with the function, or you notice the behavior has changed and not for the better, let me know.  I’ve found in the last week that people made heavy use of some undocumented behavior (and by undocumented, I mean, completely unintentional), and as long as it makes sense, I’ll formalize the support.

The second reason is that I think a lot of times, we can let things go and stop innovating because things work and we don’t necessarily have the tools to move it forward.  You may not print it out as plainly as I did in the source code – but it definitely happens and when it does, services stagnate and we get, maybe, a little too attached to the status quo.  Making changes in these circumstances is hard, its painful, and it sometimes means taking a step backward…but in the end, they can provide a much clearer path to future innovation.

–tr

 Posted by at 10:37 am

  3 Responses to “MarcEdit Swap Field function updates”

  1. Hi Terry, can’t thank you enough for all your work on MarcEdit. Am back in libraries (in Iraq, of all places) after three years with an international NGO in Pakistan. I find MarcEdit to be an indispensable tool as I try to establish basic library systems and data flows here. Congrats on your new job, btw. — Glenda

  2. Hello Terry,

    I am an avid user of the Swap Field function and I am finding that since the rewrite, I can’t seem to get the Swap Field function to perform a second swap. I tend to have to move the holdings fields around before importing records.

    Today, I am struggling with records again. I moved one subfield from the 852 to a new 952 field. When the process finished, I attempted to move a different subfield from the 852 field to the 952 field that was just created. I checked off the “Add to Exisitng Field” box and the results come back a zero. Nothing gets moved. If I attempt it again but check off the “Add to Exisitng/Create New” box, the process will work, but the second subfield is moved to a second instance of the 952 field, so the function has created another 952 field in each record rather than placing that second subfield into the original 952 field I created.

    I’ve tried it with two different files of MARC records and it is happening with both — I can move one subfield to a newly created 952 field, but when I try to go for a second swap to that new 952 field, nothing moves.

    Is it a bug or do I need to do something new to get it to swap something else?

    Thanks,

    Nora Blake

    • Nora,

      I took a look at the problem this morning and I cannot recreate it. It works as I would expect, I can swap data, using as many operations, against new and newly created fields — moving data from the same field into a newly swapped field or multiple fields into the newly swapped field. I’m happy to look at this further, but I’ll need to have a set of data and the specific steps you are using to see if I can recreate the problem. So, if you want to look at this further, just send me a private email with the above information.

      –tr

Sorry, the comment form is closed at this time.