MarcEdit Linked Data Rules Files Enhancements

By reeset / On / In BibFrame, MarcEdit

I’m working on a couple enhancements to the MarcEdit Linked Data rules file to accommodate proposals being generated by the PCC regarding the usage of the $0 and the $1 in MARC21 records.  Currently, MarcEdit has the ability to generate $0s for any controlled data within a MARC record, so long as a webservice for the vocabulary has been profiled in the MarcEdit rules file.  This has allowed folks to begin embedding URIs into their MARC records…but one of the areas of confusion is where services like VIAF fit into this equation.

The $0, as described in MARC, should represent the control number or URI to the controlled vocabulary.  This will likely never (or in rare cases ever be), VIAF.  VIAF is an aggregator, a switching point, an aggregation of information and services about a controlled term.  It’s also a useful service (as are other aggregators), and this had led folks to start adding VIAF data into the $0.  This is problematic, because it dilutes the value of the $0 and makes it impossible to know if the data being linked is to the source vocabulary or an aggregating service.

To that end, the PCC will likely be recommending the $1 for URIs that are linked data adjacent.  This would allow users to embed references to aggregations like VIAF, while still maintaining the $0 and the URI to the actual vocabulary — and I very much prefer this approach.

So, I’m updating the MarcEdit rules file to allow users to denote multiple vocabularies to query against a single MARC field, and to denote the subfield for embedding the retrieved URI data on a vocabulary by vocabulary level.  Prior, this was set as a global value within the field definition set.  This means that if a user wanted to query LCNAF for main entry (in the 100) and then VIAF to embed a link in a $1, they would just need to use:

<field type=”bibliographic”>
<tag>100</tag>
<subfields>abcdqnpt</subfields>
<uri>0</uri>
<special_instructions>personal_name</special_instructions>
<vocab subfield=”0″>naf</vocab>
<vocab subfield=”1″>viaf</vocab>
</field>

The Subfield Attribute now defines per vocabulary element, where the URI should be stored, and the uri element, continues to act as the global subfield setting, if not value is defined in the vocab element.  Practically, this means that if you had the following data in your MARC record:

=100  1\$6880-01$aHu, Zongnan,$d1896-1962,$eauthor.

The tool will now automatically (with the rules file updates), generate both a $0 and $1.

=100  1\$6880-01$aHu, Zongnan,$d1896-1962,$eauthor.$0http://id.loc.gov/authorities/names/n84029846$1http://viaf.org/viaf/70322743

But users could easily add other data sources if they have interests beyond VIAF.

These changes will be available in all versions of MarcEdit when the next update is posted in a week or so.  This update will also include an updated rules file that will embed viaf elements in the $1 for all main entry content (when available).

 

–tr

 

 

MarcEdit Update Notes

By reeset / On / In BibFrame, MarcEdit

MarcEdit Update: All Versions

Over the past several weeks, I’ve been working on a wide range of updates related to MarcEdit. Some of these updates have dealt with how MarcEdit handles interactions with other systems, some of these updates have dealt with integrating the new bibframe processing into the toolkit, and some of these updates have been related to adding more functionality around the programs terminal programs and SRU support. In all, this is a significant update that required the addition of ~20k lines of code to the Windows version, and almost 3x that to the MacOs version (as I was adding SRU support). In all, I think the updates provide substantial benefit. The updates completed were as follows:

MacOS:

* Enhancement: SRU Support — added SRU support to the Z39.50 Client
* Enhancement: Z39.50/SRU import: Direct import from the MarcEditor
* Enhancement: Alma/Koha integration: SRU Support
* Enhancement: Alma Integration: All code needed to add Holdings editing has been completed; TODO: UI work.
* Enhancement: Validator: MacOS was using older code — updated to match Windows/Linux code (i.e., moved away from original custom code to the shared validator.dll library)
* Enhancement: MARCNext: Bibframe2 Profile added
* Enhancement: BibFrame2 conversion added to the terminal
* Enhancement: Unhandled Exception Handling: MacOS handles exceptions differently — I created a new unhandled exception handler to make it so that if there is an application error that causes a crash, you receive good information about what caused it.

Couple of specific notes about changes in the Mac Update.

Validation – the Mac program was using an older set of code that handled validation. The code wasn’t incorrect, but it was out of date. At some point, I’d consolidated the validation code into its own namespace and hadn’t updated these changes on the Mac side. This was unfortunate. Anyway, I spent time updating the process so the all versions now share the same code and will receive updates at the same pace.

SRU Support – I’m not how I missed adding SRU support to the Mac version, but I had. So, while I was updating ILS integrations to support SRU when available, I added SRU support to the MacOS.

BibFrame2 Support – One of the things I was never able to get working in MarcEdit’s Mac version was the Bibframe XQuery code. There were some issues with how URI paths resolved in the .NET version of Saxon. Fortunately, the new bibframe2 tools don’t have this issue, so I’ve been able to add them to the application. You will find the new option under the MARCNext area or via the command-line.

Windows/Linux:

* Enhancement: Alma/Koha integration: SRU Support
* Enhancement: MARCNext: Bibframe2 Profile added
* Enhancement: Terminal: Bibframe2 conversion added to the terminal.
* Enhancement: Alma Integration: All code needed to add Holdings editing has been completed; TODO: UI work.
Windows changes were specifically related to integrations and bibframe2 support. On the integrations side, I enabled SRU support when available and wrote a good deal of code to support holdings record manipulation in Alma. I’ll be exposing this functionality through the UI shortly. On the bibframe front, I added the ability to convert data using either the bibframe2 or bibframe1 profiles. Bibframe2 is obviously the default.

With both updates, I made significant changes to the Terminal and wrote up some new documentation. You can find the documentation, and information on how to leverage the terminal versions of MarcEdit at this location: The MarcEdit Field Guide: Working with MarcEdit’s command-line tools

Downloads can be picked up through the automated updating tool or from the downloads page at: http://marcedit.reeset.net/downloads

MarcEdit LibHub Plug-in

By reeset / On / In BibFrame, MarcEdit

As libraries begin to join and participate in systems to test Bibframe principles, my hope is that when possible, I can provide support through MarcEdit to provide these communities a conduit to simplify the publishing of information into those systems.  The first of these test systems is the Libhub Initiative, and working with Eric Miller and the really smart folks at Zepheira (http://zepheira.com/), have created a plug-in specifically for libraries and partners working with the LibHub initiative.  The plug-in provides a mechanism to publish a variety of metadata formats into the system – from MARC, MARCXML, EAD, and MODS data – the process will hopefully help users contribute content and help spur discussion around the data model Zepheira is employing with this initiative.

For the time being, the plug-in is private, and available to any library currently participating in the LibHub project.  However, my understanding is that as they continue to ramp up the system, the plugin will be made available to the general community at large.

For now, I’ve published a video talking about the plug-in and demonstrating how it works.  If you are interested, you can view the video on YouTube.

 

–tr

MarcEdit’s MARCNext: JSON Object Viewer

By reeset / On / In BibFrame, MarcEdit

As I noted in my last post (http://blog.reeset.net/archives/1359), I’ll be adding a new area to the MarcEdit application called MARCNext.  This will be used to expose a number of research tools for users interested in working with BibFrame data.  In addition to the BibFrame Testbed, I’ll also be releasing a JSON Object Viewer.  The JSON Object Viewer is a specialized viewer designed to parse JSON text and provide an object visualization of the data.  The idea is that this tool could be utilized to render MARC data translated into Bibframe as JSON for easy reading.  However, I’m sure that there will be other uses as well.  I’ve tried to keep the interface simple.  Essentially, you point the tool at a JSON file and the tool will render the file as objects.  From there, you can search and query the data, view the JSON file in Object or Plain text mode, and ultimately, copy data for use elsewhere. 

image

Some additional testing needs to be done to make sure the program works well when coming across poorly formed data – but this tool will be a part of the next update.

–tr

MarcEdit’s Research Toolkit – MARCNext

By reeset / On / In BibFrame, MarcEdit

While developing MarcEdit 6, one of the areas that I spent a significant amount of time working on was the MarcEdit Research Toolkit.  The Research Toolkit is an easter egg of sorts – it’s a set of tools and utilities that I’ve developed to support my own personal research interests around library metadata – specifically, around the future of library metadata including topics the current BibFrame testing and linked data.  I’ve kept these tools private because they tend to not be fully realized concepts or ideas and have very little in the way of a user interface.  Just as important, many of these tools represent work being created to engage in the conversation that the library community is having around library metadata formats and standards, so things can and do change or drop out of the conversation and are then removed from my toolkit.

While developing MarcEdit 6, one of the goals of the project was to find a way to make some or parts of these tools available to the general MarcEdit community.  To that end, I’ll be making a new area available within MarcEdit called MARCNext.  MARCNext will provide a space to make proof of concept tools available for anyone to use, and offer a simple to use interface that anyone can use to test new bibliographic concepts like BibFrame. 

Presently, I’m evaluating my current workbench to see which of the available tools can be made public.  I have a handful that I think may be applicable – but will need some time to move them from concept to a utility for public consumption.  With that said, I will be making one tool immediately available as part of the next MarcEdit update, and that will be the BibFrame Testbed.  This is code that utilizes the LC XQuery files being developed and distributed at: https://github.com/lcnetdev/marc2bibframe with a handful of changes made to provide better support within MarcEdit.  These are my base files that will enable librarians to easily model their MARC metadata in a variety of serializations.  And using this initial work, I’ll likely add some additional serializations to the list. 

I have two goals for making this particular tool available.  First and foremost, I would like to enable anyone that is interested the ability to take their existing library metadata and model it using Bibframe concepts.  Currently, Library of Congress makes available a handful of commandline tools that users can utilize to process their metadata – but these tools tend to not be designed for the average user.  By making this information available in MarcEdit – I’m hoping to lower the barrier so that anyone can model their data and then engage in the larger discussion around this work. 

Secondly, I’m currently engaging in some work with Zepheira and other early implementers to take Bibframe testing mainstream.  Given the number of users working with MarcEdit, it made a lot of sense to provide tools to support this level of integration.  Likewise, by taking the time to move this work from the concept stage, I’ve been able to develop the start of a framework around these concepts. 

So how is this going to work?  On the next update, you will see a new link within the Main MarcEdit Window called MARCNext. 

image 
MarcEdit Main Window

Click on the MARCNext link, and you will be taken to the public version of the Research Toolkit.  At this point, the only tool being made publically available is the BibFrame Testbed, though this will change.

image 
MarcEdit’s MARCNext Window

Selecting the BibFrame Testbed initializes a simple dialog box to allow a user to select from a variety of library metadata types and convert them using BibFrame principles into a user-defined serialization. 

image 
BibFrame Testbed window

As noted above, this test bed will be the first of a handful of tools that I will eventually be making available.  Will they be useful to anyone – who knows.  Honestly, the questions that these tools are working to answer are not ones that come up on the list serv, and at present, aren’t going to help much in one’s daily cataloging work.  But hopefully they will enable every cataloger that wants to, the ability to engage with some of these new metadata concepts and at least take their existing data and see how it may change utilizing different serializations and concepts.

Questions – feel free to ask.

–tr