Jun 192015
 

Logistics

Time: 6:00 – 7:30 pm, Friday, June 26, 2015
Place: Marriott Marquis (map)
Room: Pacific H, capacity: 30

Description:

The MarcEdit user community is large and diverse and honestly, I get to meet far too few community members.  This meeting has been put together to give members of the community a chance to come together and talk about the development road map, hear about the work to port MarcEdit to the Mac, and give me an opportunity to hear from the community.  I’ll talk about future work, areas of potential partnership, as well as hearing from you what you’d like to see in the program to make your metadata live’s a little easier.  If this sounds interesting to you — I really hope to see you there.

Acknowledgements:

A *big* thank you to John Chapman and OCLC for allowing this to happen.  As folks might guess, finding space at ALA can be a challenging and expensive endeavor so when I originally broached the idea with OCLC, I had pretty low expectations.  But they truly went above and beyond any reasonable expectation, working with the hotel and ALA so this meeting could take place.  And why they didn’t ask for it — they have my personal thanks and gratitude.  If you can attend the event, or heck, wish you could have but your schedule made it impossible — make sure you let OCLC know that this was appreciated.

 Posted by at 1:24 pm
Jun 062015
 

Having made one preview available for evaluation and feedback, I’ve been diligently working on updating the tool and working on new functionality.  This includes moving onto developing a new notification service so that preview users know when new builds are available and providing a new preferences window to enable support for changing applicable preferences currently exposed within the application.  From the perspective of new work — I’ve begun working on the MarcEditor.  At this point, I’ve mocked out the window and am starting to create the global editing toolsets and connecting actions to the various UI elements.  This is going to be a bit of a time consuming process — but thankfully, it’s been made somewhat easier by the fact that much of the code within the MarcEditor that’s not platform specific, has been moved outside the application to re-usable assemblies.  There’s a bit more refactoring that needs to be done, and I’ll need to re-think how the program streams data into the MarcEditor edit window since the OSX apis make this a bit difficult — but it’s getting there.  Below — you will find screenshots of some of the new work?

–tr

Mac Port Notification Example

MarcEdit Mac Port Notifications Window Example

MarcEdit Mac Preferences: MARCEngine

MarcEdit Mac Preferences window: Current preferences exposed for update are for the MARCEngine and the Automatic Update notification.

This is the initial MarcEditor Wireframe for the Mac.  This feels pretty solid at this point.

This is the initial MarcEditor Wireframe for the Mac. This feels pretty solid at this point.

Current options in the MarcEditor scheduled for the first release

Current options in the MarcEditor scheduled for the first release

MarcEdit Mac MarcEditor Edit Menu wireframe -- options targeted for the first release

MarcEdit Mac MarcEditor Edit Menu wireframe — options targeted for the first release

MarcEdit Mac MarcEditor Reports Menu wireframe showing functions targeted for the first release

MarcEdit Mac MarcEditor Reports Menu wireframe showing functions targeted for the first release

MarcEdit Mac MarcEditor Tools Menu wireframe showing functions targeted for the first release

MarcEdit Mac MarcEditor Tools Menu wireframe showing functions targeted for the first release

May 292015
 

MarcEdit provides lots of different ways for users to edit their data.  However, one use case that comes up often is the ability to perform an action on a field or fields based on the presence of data within another field.  While you can currently do this in MarcEdit by using tools to isolate the specific records to edit, and then working on just those items — more could be done to make this process easier.  So, to that end, I’ve updated the Replace Function to include a new conditional element that will allow MarcEdit to presort using an in-string or regular expression query, prior to evaluating data for replacement.  Here’s how it will work…

When you first open the Replace Window:

Replace Function Changes

Notice that the conditional string text has been replaced.  This was confusing to folks – because maybe that didn’t reflect exactly what was being done.  Rather, this is an option that allows a user to run an instring or Regular Expression search across your entire record before the Find/Replace is run.  The search options grouped below – these *only* affect the Find/Replace textboxes.  They do not affect the options that are enabled when the Perform Find/Replace If…is checked.  Those data fields have their own toggles for instring (has) or regular expression (regex) matching.

 

If you check the box, the following information will be displayed:

New Replace Functionality

Again – the If  [Textbox] [REGEX] is a search that is performed and must evaluate as true in order for the paired find and replace runs.  The use case for this function are things like:

  • I want to modify the field x but only if foobar is found in field y.

 

There are other ways to do this by extracting data from files and creating lots of different files for processing or writing a script – but this will give users a great deal more flexibility when wanting to perform options, but only if specific data is found within a field.

 

A simple example would be below:

Example of the new Replace

This is a non-real world example of how this function works.  A user wants to change the 050 field to an 090 field, but only if the data in the 945$a is equal to an m-z.  That’s what the new option allows.  By checking the Perform Find/Replace If option, I’m allowed to provide a pre-search that will then filter the data sets that I’m going to actually perform the primary Find/Replace pair on.  Make sense?  I hope so.

Finally – I’ve updated the code around the task wizard so that this information can be utilized within tasks.  This enhancement will be in the next available update.

–tr

 Posted by at 11:09 pm
May 252015
 

I’ve been working hard on making a few changes to a couple of the MarcEdit internal components to improve the porting work.  To that end, I’ve posted an update that targets improvements to the Deduping and the Merging tools.

Updates:

  • Update: Dedup tool — improves the handling of qualified data in the 020, 022, and 035.
  • Update: Merge Records Tool — improves the handling of qualified data in the 020, 022, and 035.

Downloads can be picked up using the automated update tool or by going to: http://marcedit.reeset.net/downloads/

–tr

 Posted by at 11:26 pm
May 062015
 

I’ve posted an update.  The change log is below:

  • Enhancement: Dedup Function: Added support for multiple field evaluation. 
  • Enhancement: Dedup Function: Changed default control number field to 001|019$a
  • Enhancement: Merge Function: Added 019 to the evaluation profile
  • Enhancement: Updated the 001, 010$a, 020$a, 022$a, 035$a process so that it utilizes the same process as the MARC21 process.  This adds multiple field support when handling these individual elements.
  • Optimizations: MARCEngine: character conversion code has been moved into the MARCEngine.
  • Optimizations: Dedup code has been moved into the meedit assembly.
  • Update: JSON View JSON processing assembly has been updated.
  • Update: MARCEngine SAXON dependency has been updated.
  • Behavior Change: MARC Tools UI has been updated to utilize a hybrid of a menu and toolbar.
  • Bug Fix: Windows XP Support has been restored to the installer.

You can pick up the update either using the automated update tool or by going to: http://marcedit.reeset.net/downloads/

–tr

 Posted by at 8:11 pm
Apr 112015
 

This all started with a conversation over twitter (https://twitter.com/_whitni/status/583603374320410626) about a week ago.  A discussion about why the current version of MarcEdit is so fragile when being run on a Mac.  The short answer has been that MarcEdit utilizes a cross platform toolset when building the UI which works well on Linux and Windows systems, but tends to be less refined on Mac systems.  I’ve known this for a while, but to really do it right, I’d need to develop a version of MarcEdit that uses native Mac APIs, which would mean building a new version of MarcEdit for the Mac (at least, the UI components).  And I’ve considered it – mapped out a road map – but what’s constantly stopped me has been a lack of interest from the MarcEdit community and a lack of a Mac system.  On the community-side, I can count on two hands the number of times I’ve had someone request a version of MarcEdit  specifically for a Mac.  And since I’ve been making a Mac App version of MarcEdit available – it’s use has been fairly low (though this could be due to the struggles noted above).  With an active community of over 20,000, I try to put my time where it will make the most impact, and up until a week ago, better support for Mac systems didn’t seem to be high on the list.  The second reason is I don’t own a Mac.  My technology stack is made up of about a dozen Windows and Linux systems embedded around my house because they play surprisingly well together, where as, Apple’s walled garden just doesn’t thrive within my ecosystem.  So, I’ve been waiting and hoping that the cross-platform toolset would get better and that in time, this problem would eventually go away.

I’m giving that background because apparently I’ve been misreading the MarcEdit community.  As I said, this all started with this conversation on twitter (https://twitter.com/_whitni/status/583603374320410626) – and out of that, two co-conspirators, Whitni Watkins and Francis Kayiwa set out to see just how much interest there actually was in having dedicated version of MarcEdit for the Mac.  The two set out to see if they could raise funds to acquire a Mac to do this development and indirectly, demonstrate that there was actually a much larger slice of the community interested in seeing this work done.  And, so, off they went – and I set back and watched.  I made a conscious decision that if this was going to happen, it was going to be because the community wanted it and in that, my voice wasn’t necessary.  And after 8 days, it’s done.  In all, 40 individuals contributed to the campaign, but more importantly to me, I heard directly from around 200+ individuals that were hopeful that this project would proceed. 

Development Roadmap

Now the hard work starts.  MarcEdit is a program that has been under constant development since 1999 – so even just rewriting the UI components of the application will be a significant undertaking.  So, I’m breaking up this work in chunks.  I figure it would take approximately 8-12 months to completely port the UI, which is a long-time.  Too long…so I’m breaking the development into 3 month “sprints”.  the first sprint will target the 80%, the functionality that would make MarcEdit productive when doing MARC editing.  This means porting the functionality for all the resources found in the MARC Tools and much of the functionality found in the MarcEditor components.  My guess is these two components are the most important functional areas for catalogers – so finishing those would allow the tool to be immediately useful for doing production cataloging and editing.  After that – I’ll be able to evaluate the remainder of the program and begin working on functional parity between all versions of the application. 

But I’ll admit, at this point, the road map is somewhat even cloudy to me.  See, I’ve written up the following document (http://1drv.ms/1ake4gO) and shared it with Whitni and asked her to work with other Mac users to refine the list and let me know what falls into that 80%.  So, I’ll be interested to see where their list differs from my own.  In the mean time, I’ll be starting work on the port – creating wireframes and spending time over the next week hitting the books and familiarizing myself with Apple’s API docs and the UI best practices (though, I will be trying to keep the program looking very familiar to the current application – best practices be damned).  Coding on the new UI will start in earnest around May 1 – and by August 1, 2015, I hope to have the first version built specifically for a Mac available.  For those interested in following the development process – I’ll be creating a build page on the MarcEdit website (http://marcedit.reeset.net) and will be posting regular builds as new areas of the application are ported so that folks can try them, and give feedback. 

So, that’s where this stands and this point.  For those interested in providing feedback, feel free to contact me directly at reeset@gmail.com.  And for those of you that reached out or participated in the campaign to make this happen, my sincere thanks. 

–TR

MarcEdit 101 Webinar Series

 MarcEdit  Comments Off on MarcEdit 101 Webinar Series
Mar 312015
 

The MarcEdit 101 Webinar Series were created over the course of multiple months for the CARLI (http://www.carli.illinois.edu/) consortium in Spring 2015.  In late March 2015, CARLI reached out to me and requested that these webinars be made available to the larger MarcEdit community, so if you find these webinars useful, please reach out and thank the folks at CARLI.

Couple of notes, these webinars are being made available as is, save for the following modifications:

  1. Attendee names have been anonymized.  While I’m certain most attendees would have no problem with their names showing up in these webinar lists, the original intended audience was locally scoped to CARLI and it’s members.  Masking attendees was done primarily because of this change of scope.
  2. The Q/A at the end of the sessions has generally been removed from the webinars.  Again, these are localized webinars and questions asked during the webinars tend to be within the scope of this consortia.

I’ll be making these video available over the next couple of months.  Again, if you find these webinars useful, please make sure you let the folks at CARLI know.

Series URL: http://marcedit.reeset.net/marcedit-101-workshop

–TR

MarcEdit 6.0 Update

 MarcEdit  Comments Off on MarcEdit 6.0 Update
Mar 172015
 

List of changes below:

** Bug Fix: Delimited Text Translator: Constant data, when used on a field that doesn’t exist, is not applied.  This has been corrected.
** Bug Fix: Swap Field Function: Swapping control field data (fields below 010) using position + length syntax (example 35:3 to take 3 bytes, starting at position 35) not functioning.  This has been corrected.
** Enhancement: RDA Helper: Checked options are now remembered.
** Bug Fix: RDA Helper: Abbreviation Expansion timing was moved in the last update.  Moved back to ensure expansion happens prior to data being converted.
** Enhancement: Validator error message refinement.
** Enhancement: RDA Helper Abbreviation Mapping table was updated.
** Enhancement: MarcEditor Print Records Per Page — program will print one bib record (or bib record + holdings records + authority records) per page.
** Bug Fix: Preferences Window: If MarcEdit attempts to process a font that isn’t compatible, then an error may be thrown.  A new error trap has been added to prevent this error.

You can get the new download either through MarcEdit’s automatic update tool, or by downloading the program directly from: http://marcedit.reeset.net/downloads

–tr

 Posted by at 7:20 pm

Conditional Regular Expression Replacements using substitutions in MarcEdit

 MarcEdit  Comments Off on Conditional Regular Expression Replacements using substitutions in MarcEdit
Mar 112015
 

A question that comes up occasionally is the need to be able to conditionally add or replace a set of character data within a MARC Field.  For example, consider this use case:

I’d like to add a period to the end of a field (like say, the 650 field), but only under the following conditions:

  1. The field ends with a word (a-z) character.
  2. The field doesn’t already end in a period or parenthesis
  3. If the field ends with any other punctuation, that value is replaced with a period.

Doing option 1 and 2 is easy and straightforward.  For that option, I’d probably do something like this:

Find: (=650.*[^.;])$
Replace With: $1.

This allows MarcEdit to match any line that doesn’t end in a period or a parenthesis.  However, the conditional makes this more difficult.  In C#’s implementation of regular expressions, you can use substitutions and conditional matching to achieve the above result.  Consider the following data:

=650  \6$aMusique populaire$zQuébec (Province)$y1951-1960.
=650  \6$aMusique populaire$zQuébec (Province)$y1961-1970,
=650  \6$aMusique populaire$zQuébec (Province)$y1961-1970)
=650  \6$aMusique populaire$zQuébec (Province)$y1961-1970;
=650  \6$aMusique populaire$zQuébec (Province)$y1961-1970
=650  \6$aMusique populaire$zQuébec

Using the above criteria, I’d like to be able to run a process that will turn the comma in line two, into a period, the semi-colon in line 4 into a period and add a period to the end of line 5 and 6.  To do this, you’d setup a substitution. 

Find: ((?<one>=650.*[\w])|(?<one>=650.*)(?<two>[^.)]))$
Replace With: ${one}.

So what exactly is happening here.  In the .NET regular expressions, you can use named substitutions to represent groups.  In this case, we create a conditional using an ‘or’ clause, using the same substitution name for each element of the clause.  We then push out the replacement clause and give it a separate grouping.  Now, we have isolated the data we want to keep, and can use the same statement to get all the data we want to keep/append to.  Using the above, you will receive the following output:

=650  \6$aMusique populaire$zQuébec (Province)$y1951-1960.
=650  \6$aMusique populaire$zQuébec (Province)$y1961-1970.
=650  \6$aMusique populaire$zQuébec (Province)$y1961-1970)
=650  \6$aMusique populaire$zQuébec (Province)$y1961-1970.
=650  \6$aMusique populaire$zQuébec (Province)$y1961-1970.
=650  \6$aMusique populaire$zQuébec.

Obviously, the above is a fairly simple example – but the concept should can be applied to much more complicated workflows.  If you are interested in reading more about the Regular Expression implementation used in MarcEdit, please see: https://msdn.microsoft.com/en-us/library/vstudio/az24scfc(v=vs.100).aspx.

Questions, let me know.

–tr

 Posted by at 9:19 am
Feb 232015
 

A new version of MarcEdit has been made available.  The update includes the following changes:

  • Bug Fix: Export Tab Delimited Records: When working with control data, if a position is requested that doesn’t exist, the process crashes.  This behavior has been changed so that a missing position results in a blank delimited field (as is the case if a field or field/subfield isn’t present.
  • Bug Fix: Task List — Corrected a couple reported issues related to display and editing of tasks.
  • Enhancement: RDA Helper — Abbreviations have been updated so that users can select the fields that abbreviation expansion occurs.
  • Enhancement: Linked Data Tool — I’ve vastly improved the process by which items are linked. 
  • Enhancement: Improved VIAF Linking — thanks to Ralp LeVan for pointing me in the right direction to get more precise matching.
  • Enhancement: Linked Data Tool — I’ve added the ability to select the index from VIAF to link to.  By default, LC (NACO) is selected.
  • Enhancement: Task Lists — Added the Linked Data Tool to the Task Lists
  • Enhancement: MarcEditor — Added the Linked Data Tool as a new function.
  • mprovements: Validate ISBNs — Added some performance enhancements and finished working on some code that should make it easier to begin checking remote services to see if an ISBN is not just valid (structurally) but actually assigned.
  • Enhancement: Linked Data Component — I’ve separated out the linked data logic into a new MarcEdit component.  This is being done so that I can work on exposing the API for anyone interested in using it.
  • Informational: Current version of MarcEdit has been tested against MONO 3.12.0 for Linux and Mac.

Linked Data Tool Improvements:

A couple specific notes of interest around the linked data tool.  First, over the past few weeks, I’ve been collecting instances where id.loc.gov and viaf have been providing back results that were not optimal.  On the VIAF side, some of that was related to the indexes being queried, some of it relates to how queries are made and executed.  I’ve done a fair bit of work added some additional data checks to ensure that links occur correctly.  At the same time, there is one known issue that I wasn’t able to correct while working with id.loc.gov, and that is around deprecated headings.  id.loc.gov currently provides no information within any metadata provided through the service that relates a deprecated item to the current preferred heading.  This is something I’m waiting for LC to correct.

To improve the Linked Data Tool, I’ve added the ability to query by specific index.  By default, the tool will default to LC (NACO), but users can select from a wide range of vocabularies (including, querying all the vocabularies at once).  The new screen for the Linked Data tool looks like the following:

image

In addition to the changes to the Linked Data Tool – I’ve also integrated the Linked Data Tool with the MarcEditor:

image

And within the Task Manager:

image

The idea behind these improvements is to allow users the ability to integrate data linking into normal cataloging workflows – or at least start testing how these changes might impact local workflows.

Downloads:

You can download the current version buy utilizing MarcEdit’s automatic update within the Help menu, or by going to: http://marcedit.reeset.net/downloads.html and downloading the current version.

–tr

 Posted by at 9:38 pm