Christmas 2007

 Family  Comments Off
Dec 262007
 

Christmas has come and in 10 minutes, will go for another year, and it was fun.  Nathan came up with two of my favorite memories of this year’s Christmas.  One happened two days ago when we opened presents at my parents house.  Nathan and Kenny got a little foam shooter and he was hot to give it a try.  I wasn’t  paying attention, and I kid you not, I hear the following next to me.

Click (Nathan shoots off a foam disk)…Ouch!.  I looked over at him and he was squinting looking at me and said: I shot my eye out.  Of course, he didn’t shoot his eye out, but in classic Christmas Story fashion, he’d shot one of the disk near his eye.  Then, not 10 minutes later, Kenny does the same thing.  We had to put that one away. :)

I’ll have to admit, I love the Christmas Story, so I had a hard time not laughing while I was making sure Nathan was ok.  Then today, Santa had delivered Kenny and Nathan their presents.  See, we open presents from Mom and Dad Christmas eve and then the boys get to see if Santa brings them anything on Christmas.  I’m not sure how many more years we have where Kenny will still get into this, but this is Nathan’s first year of really knowing who Santa is and why he should care.  We’d spent the day tracking Santa on the Norad Tracks Santa site (a wonderful site for kids on Christmas), and the kids were bouncing off the roof when we finally put them to bed.  Of course, they got up early (or I stayed up too late, not sure which), and ran downstairs.  Kenny started passing out the presents and to his delight, he found that Santa had brought him some presents, Nathan some presents and he found a present for Alyce under the tree as well.  I’d put one down last night to save for Christmas.  Well, Nathan thought this was hilarious.  He was looking over the presents and seen that I didn’t get anything and asked, Dad, did you get any presents?  I said no, and he got a mischievous little smile on his face and said, Dad, were you naughty.  And I said, maybe.  At which point, he happily started dancing around the house, reveling in my naughtiness.  I guess Alyce told him later that I had given her the present and that Santa only brings presents for little boys and girls, but it made me laugh a bit.

Anyway, Christmas was great.  We had a good time with family, helped my in-laws lay some tile in their kitchen and even had some snow (for the first time on Christmas that I can remember).  None of the snow stuck, but snow is snow.  :)

I hope everyone was able to take some time, relax with family and friends and had a merry Christmas. 

 

–TR

 Posted by at 1:11 am
Dec 262007
 

I took a couple of minutes and made a few changes to MarcEdit and the OCLC plug-in to provide some additional functionality to the plug-in framework and fix an error in the OCLC plug-in. 

Changes:

MarcEdit:

  • One real change.  In the MacroInterfaces.dll (the library that allows the Scripting interface and the Plug-in interface access to the MarcEditor and its functions) I’ve added two new functions: AddButton and RemoveButton.  These two functions allow users to have plug-ins place buttons on the toolbar of the application (at least, on specific windows). 

OCLC Plug-in:

  • I’ve added the code to initiate a button and interact when it is clicked to the MarcEditor toolbar.
  • I’ve corrected the 007 error.  See: http://blog.reeset.net/archives/480

So what will you see with the changes.  Well, the big change you will see is when you initialize the plug-in in the MarcEditor.  Once you have downloaded the update to both MarcEdit and the Plug-in (you need both), you will see the following when the plug-in is executed:

image

The new button added has been highlighted.  This button now acts as the new Save button when you have made your changes to the OCLC data records.  This will move the data back into the OCLC Save File.  Remember — at this point, you will want to make a backup of your Save Files before you make your changes — just in case there are other fields in OCLC’s XML format that are different than I would have expected. 

If you have downloaded the OCLC Plug-in and would like to update it.  At this point, the process isn’t as streamlined as I’d like (I’ll fix that this week while I’m taking some time off to recharge).  Essentially, you need to Uninstall the plug-in (using the plug-in manager or, delete the oclc_helper.dll from the marcedit plug-ins directory (generally, c:\program files\marcedit 5.0\plugins\).  If you uninstall with the plug-in manager, you will need to close and restart MarcEdit.  Then open the plug-in manager and download the new plug-in.  If you delete the library directly from the plug-ins directory — just open MarcEdit, select the plug-in manager and download the plug-in.

I’ve updated a new version of MarcEdit: MarcEdit51_Setup.exe.  I’ve also uploaded a new version of the plug-in (download this through plug-in manager [see above]).  Source can be downloaded from: oclc_helper.zip.

If you have a strong desire to see how this type of interaction is accomplished in C#, please see the following post and sample project file: http://blog.reeset.net/archives/481.

 

–TR

 Posted by at 12:41 am
Dec 242007
 

Example Project Source: PluginProject.zip

Because I’ve been doing a lot of work with MarcEdit and plug-ins, I thought I’d post some sample code for anyone interested in how this might work.  Essentially, the sample project includes 3 parts — a host application, a set of Interfaces and a Shared library.  Making this work requires a couple of important parts. 

First, the host application (either the form or class), need to implement the set of interfaces.  So for example, if interaction with a form in the hosted application was need, you would configure the form to implement a set of interfaces.  This would look like:

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Text;
using System.Windows.Forms;

namespace HostApp
{
public partial class Form1 : Form, HostInterfaces.IHost
{
//...
}

This implements the IHost class (link to msdn) — a generic class that allows you to

pass objects between dynamically loaded libraries.  .NET includes a IScript interface that allows for scripting functionality as well. 

Anyway, the interfaces are simply like delegates — they define the visible functions/methods that will be accessible to a foreign assembly.  This is the simpliest file to create.  It looks something like this:


using System;
using System.Collections.Generic;
using System.Text;

namespace HostInterfaces
{
public interface IHost
{
System.Windows.Forms.Label label { get;}
System.Windows.Forms.ToolStripButton AddButton(string caption);
void RemoveButton(System.Windows.Forms.ToolStripButton t);

}
}

Finally, the Dynamic assembly has the ability to work with any function/object within the host application that has been made public through the interface.  For this sample project, I’ve shown how to modify a label (on the host application), add a button to a toolbar and respond to click events from that button. 

The project is a simple one — but should go a long way towards showing how this works.

Cheers,

–TR

Technorati Tags: ,,

 Posted by at 8:57 pm
Dec 212007
 

As I’d noted previously (http://blog.reeset.net/archives/479), some early testers had found that the Connexion plug-in that I’d written for MarcEdit stripped the 007.  I couldn’t originally figure out why — it’s just a control field and their syntax for control fields is pretty straightforward.  However, after looking at a few records with 007 records, I could see why.  In Connexion, OCLC lets folks code the 007 using delimiters like a normal variable MARC field (when its not) — and they save it as such — using delimiters.  For example:

<v007 i2=" " i1=" " im="0">
  <sa>
    <d>s</d>
  </sa>
  <sb>
    <d>d</d>
  </sb>
  <sd>
    <d>f</d>
  </sd>
  <se>
    <d>s</d>
  </se>
  <sf>
    <d>n</d>
  </sf>
  <sg>
    <d>g</d>
  </sg>
  <sh>
    <d>n</d>
  </sh>
  <si>
    <d>n</d>
  </si>
  <sj>
    <d>z</d>
  </sj>
  <sk>
    <d>u</d>
  </sk>
  <sl>
    <d>u</d>
  </sl>
  <sm>
    <d>u</d>
  </sm>
  <sn>
    <d>d</d>
  </sn>
</v007>

I’ll admit — I have no idea why they went with this format.  From my perspective, its clunky.  The 007, as a single control field, is fairly easy to parse as it can have up to 13 bytes, with number of bytes specified 0 byte of the data element.  In this format, you actually have to create 9 different templates for the different possibilities in order to account for different field lengths, byte combinations and delimiter settings.  Honestly, my first impression when looking at this was that its a perfect example of how something so simple can become much more difficult than need be.  Personally, I would have been happier had they broke from their MARCXML like syntax for this one field to create an special 007 element.  Again, this is something that could have been easily abstracted in the XSLT translation — but to be fair, I don’t think that they figured anyone but OCLC’s connexion team would ever be trying to work with this. 

So how I’m solving it?  Well, one of the cool things working with XSLT (and .NET in general) is the ability to use extensions to help fill in missing functionality in the XSLT language (in my case, the ms:script extension in the msxml library).  Since this transformation isn’t one that I’m really sharing (outside the plug-in), I’m not too worried about its portability.  So, what I’ve done is created a number of helper C# functions and embedded them within the xslt document to aid processing.  For example,

<xsl:stylesheet version="1.0"
xmlns:marc="http://www.loc.gov/MARC21/slim"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
 xmlns:ms="urn:schemas-microsoft-com:xslt"
 xmlns:osu="urn:oregonstate-edu:xslt"
 extension-element-prefixes="osu">
  <xsl:output method="xml" indent="yes" />
  <ms:script language="C#" implements-prefix="osu">
    <![CDATA[
        public int length(string s) {
          s = s.ToLower();
          if (s=="c") {
             return 14;
          } else if (s=="d") { return 6;}
          else if (s=="a") { return 8;}
          else if (s=="h") { return 13;}
          else if (s=="m") { return 10;}
          else if (s=="k") { return 6;}
          else if (s=="g") { return 9;}
          else if (s=="r") { return 11;}
          else if (s=="s") { return 14;}
          else if (s=="f") { return 10;}
          else if (s=="v") { return 9;}
          else { return 8;}
        }
      ]]>
  </ms:script>
 

This is a simple function that I’m using to track the number of elements needed for the processing template.  This is because I don’t want to create 9 different XSLT templates for each processing type, so I’m using some embedded C# to simplify the process.  On the plus side, using these embedded scripts make the translation process much faster on the .NET side (since .NET compiles xslt to byte code anyway before running any translation process), and this is a technique that I’ve never really had to use before so I was able to get a little practical experience.  Still don’t like it though.

–TR

 Posted by at 10:35 am
Dec 192007
 

Apparently folks are interested in this type of tool, as I had a few folks trying it.  One known issue and two changes coming in the next day or two. 

Known Issue:

  • For some reason, the internal OCLC metadata schema uses delimiters for the 007 field (even though its a control field).  My default template treats it as a control field so this data is lost.  Easy fix — will be corrected.

Changes:

  • MarcEdit plug-in manager — Update button.  At present, the process for updating plug-ins is remove the plug-in, restart MarcEdit, then download the plug-in, then restart.  I’ll add an update button so that the process will be update, then restart MarcEdit.
  • Automated check version function on plug-ins.

–TR

Technorati Tags: ,,
 Posted by at 2:24 am

MarcEdit 5.1 update

 MarcEdit  Comments Off
Dec 182007
 

Lots of updates.  Of note:

  1. Inclusion of a plug-in manager
    1. Necessitated the inclusion of Zip functionality (thanks to the SharpDevelop project [http://www.icsharpcode.net/OpenSource/SD/])
    2. AppDomain manager (for loading and unloading code)
  2. Fixed Preview file error: Follow the code execution patches, windows has changed the format in which file names are passed to applications.  This corrects the problems introduced by this change.
  3. Fixed Delimited Text Translator:  Essentially, I re-wrote the parse functions to provide better delimited text support.
  4. Lots of bug fixes.

BTW, I’ll be putting up a version of mantis (or something else) to provide easier bug tracking.  This way, I can point folks to my tracking system (which currently is just an access database that I use to track tickets).

The big change is obviously the Plug-in manager/functionality.  The program now supports the ability for folks to create code modules that can interact with MarcEdit.  I have ideas for two new plug-ins — one relating to the Z39.50 client and one being a Dspace plug-in to help me with spidering and packaging HTML/sites for inclusion into Dspace.   Like the OCLC Plug-in, I’ll post the source files of each plug-in as I complete them.

The source code for the OCLC Plug-in can be found at: OCLC_Plugin.zip. [update: One quick note about this particular plug-in.  I didn't get any documentation or help from OCLC, so while I'm confident enough that this works to put it out there for folks to try if they like -- please make sure that you make a backup of your Connexion file before processing.  Without the database schema, much of what I know about the Connexion database structure was through examination, and trial and error. ]

Updates for MarcEdit can be found at: MarcEdit51_Setup.exe.

If you need info on some of the update changes, see:

–TR

Dec 172007
 

So much for an early night.  It’s 12:58 am — fortunately, all the code is working on my two test systems.  The new plug-in code is working, install is working — so all good.  Now I just need to run the program through its unit tests on my clean machine.  Since that takes about an hour, there’s no way I’m doing it tonight.  So, I’ll likely send the build privately to a few folks tomorrow early — but I’ll have a link up tomorrow afternoon.

–TR

 Posted by at 2:00 am
Dec 172007
 

Quick enhancement to to the Delimited Text Translator.  The delimited text translator provides a pretty flexible flexible method for translating delimited data into MARC.  It gives you a GUI interface to profile (and save profiles) of how a delimited text file should be processed.  Of course, a number of strict rules have always had to be followed — the most inconvenient being that subfields cannot be repeated within the same field.  Mostly, this is because the original architecture didn’t have a way to determine if a subfield should just be added to an existing field or be part of a new subfield if the data was repeated.  So, I just made a hard an fast rule that there was no repeatable subfield.  If the delimiter encounted a repeated subfield, it would simply create a new field.  I removed this limitation this afternoon.   You will now find a new option, Repeatable Subfield.  If this option is marked, the tool will allow the subfield to be repeatable.  Here’s what the new interface looks like.  The new option is in the lower left:

image

 

–TR

 Posted by at 12:11 am
Dec 162007
 

Ugh, I’m hoping that I can keep my eyes open long enough to finish getting this built, but thought I’d provide a little info on the plug-in framework that I’m setting up in MarcEdit.  First, a reason why I’m even doing this. 

As time is going forward, I’m finding that I have a few ideas for functions (requested or not) that I’d like to add to MarcEdit.  However, the program is getting large enough (in terms of download size) — so I’ve developed a plug-in framework to allow others (if they like) but mostly so I can create some code modules that folks can choose to download or no.  Presently, the plug-in tools included into MarcEdit include a plug-in manager and then a way to execute plug-ins.  The last thing needing to be added is I need to override a few of the MarcEditor operations like the save function so that if you are executing a plug-in module, the module can react to save requests rather than having to have users step back out to the plug-in interface.  But if I don’t finish this tonight, I still may package and give out what I’ve got at the moment.

 

Anyway, here’s how its working.  I’ll show the only plug-in that I’ve written thus far.  I’ve wanted for a while to be able to interact with the OCLC local save files.  While not documented anywhere, a little examination of the data files shows that OCLC is using an ms access data format for their database storage and using a funny custom xml format for their own internal data storage format.  It looks a lot like MARC21XML — though not exactly.  Not sure why they didn’t go that root though — since the ISO draft of the format could have supported all the elements I’m currently seeing.  But no worries.  It’s easy enough to parse.  I’ll be posting the source to the plug-in once I get this finished, so you’ll get a chance to see how it all works then (including the MarcEdit interaction).

Plug-in Manager:

This works a lot like the Mozilla plug-in manager.  To get to the plug-in manager, you will see a new option on the Main window under tools:

image

You will also note from the above that a new menu item, Plug-ins has been added to the main window and the MarcEditor (the two parts of the program that currently support plug-in interaction).  When you click on the Plug-in Manager option, you see the following:

image

At present, I having pushed the OCLC Connexion Data plug-in up to the server, so its not in the available list — but you can see how installed and available plug-ins can be found, downloaded and removed. 

So how does it work — essentially, the plug-in should be created as a .NET assembly.  Doesn’t matter the language.  The OCLC example is written in C#, but any .NET compatible language will work (IronPython, Java#, C++, etc).  For a plug-in to work in MarcEdit, you need to create a Connection class with a Startup Function.  For example:

 

using System;
using System.Collections.Generic;
using System.Text;

namespace oclc_helper
{
    public class Connection 
    {
        
        public static void Startup(Interfaces.IHost objEditor)
        {
            //so I need to make sure I can access the macrointerfaces host.
            if (objEditor != null)
            {
                variables.objEditor = objEditor;
            }
            frmQuery frm = new frmQuery();
            frm.Show();
        }
    }
}

This is the entire code sample from the OCLC Connexion plug-in.  Essentially, a static startup function must be created.  It takes one argument — the an Interface from the MarcEdit application.  This is so you can interact with the MarcEditor.  If this plug-in isn’t setup to interact with the MarcEditor, null will be passed into the function. 

From there, its just a normal assembly.  For example, the OCLC tool current looks like the following:

Opening screen:

image

So, you can see the plug-in shows up in the new Plug-in menu entry.  Click on the entry and you see the following window:

image

 

Select the Connexion file to load, then click on Load File.  You will then see the following:

image

At this point, the plug-in is essentially just loading data directly from the MS Access Data store.  This takes data from the tblSaveFile which stores among other things, the save #, oclc number (will be set to NEW or a number) the record title, in addition to other info.

So select the data and clicking on the edit records button currently exports this data into the MarcEditor and minimizes this window.  This is because at this point, you still need to save the edits using the plug-in wizard rather than from the editor.  Should change I think, but I’m giving you how it works at this moment.

Anyway, select some items and click Edit Records.  You see this:

image

Basically, it takes the data into MarcEdit’s mnemonic format.  Of course, the data is all in Unicode (since that’s how Connexion wants to work with you).  So you edit data by using Unicode characters (not the mnemonics) [Note to self, make sure that the one mnemonic, {dollar} has been replaced when retranslated].

After you make your edits, you reinitalize the minimized plug-in window.  You will now see a save button.

image

The program makes the assumption that you having deleted any items, reorganized items — but have kept them in the same order that they were exported in.  So long as that’s the case, you click on Save Records and the program will retranslate the edited data from the MarcEditor back into Connexion’s database file. 

 

So that’s how it will work.  As I say, I’ll post the source when I push the plug-in up to the server.  With luck, that should happen tonight.  I’ve been fixing and enhancing a number of functions — so its been a long two days as I’ve been finishing up this update.  I think I’ve slept 4 hours since Friday.  Hopefully, I can wrap this up before 10 tonight — that way its not a long week next week.

 

–TR

 Posted by at 5:49 pm
Dec 122007
 

A quick note to let a few folks that are waiting know that I do have a MarcEdit update coming.  All the code has been updated and tested, its just getting a chance to build the install.  I’d hoped to do it this evening, but I’ve been busy working on other projects and just needed a break tonight.  Anyway, with luck, I should be posting in a day or so.

 

–TR

 Posted by at 1:18 am