MarcEdit Update Notes

 MarcEdit  Comments Off on MarcEdit Update Notes
Sep 012016
 

Posted Sept. 1, this update resolves a couple issues.  Particularly:

Windows/Linux:

* Bug Fix: Custom Field Sorting: Fields without the sort field may drop the LDR.  This has been corrected.
* Bug Fix: OCLC Integration: regression introduced with the engine changes when dealing with diacritics.  This has been corrected.
* Bug Fix: MSI Installer: AUTOUPDATE switch wasn’t being respected.  This has been corrected.
* Enhancement: MARCEngine: Tweaked the transformation code to provide better support for older processing statements.

Mac:

* Bug Fix: Custom Field Sorting: Fields without the sort field may drop the LDR.  This has been corrected.
* Bug Fix: MARCEngine: Regression introduced with the last update that caused one of the streaming functions to lose encoding information.  This has been corrected.
* Enhancement: MARCEngine: Tweaked the transformation code to provide better support for older processing statements.

Special Notes:

I’ll be adding a knowledge-base article, but I updated the windows MSI to fix the admin command-line added to allow administrators to turn off the auto-update feature.  Here’s an example of how this works: >>MarcEdit_Setup64.msi /qn AUTOUPDATE=no

I don’t believe the AUTOUPDATE key is case sensitive – but the documented use pattern is upper-case and what I’ll test against going forward.

Downloads are available via the downloads page: http://marcedit.reeset.net/downloads/

–tr

MarcEdit Mac Update–Inclusion of Console Mode

 MarcEdit  Comments Off on MarcEdit Mac Update–Inclusion of Console Mode
Aug 282016
 

One of the gaps in the Mac version of MarcEdit has been the lack of a console mode.  This update should correct that.  However, a couple things about how his works…

1) Mac applications are bundles, so in order to run the console program you need to run against the application bundle.  What does this look like?   From the terminal, one would run
>>/Applications/MarcEdit.app/Contents/MacOS/MarcEdit –console

The –console flag initializes the terminal application and prompts for file names.  You can pass the filenames (this must be fully qualified paths at this point) via command-line arguments rather than running in an interactive mode.  For example:
>>/Applications/MarcEdit.app/Contents/MacOS/MarcEdit –s /users/me/Desktop/source.mrc –d /users/me/Desktop/output.mrk –break

The above would break a MARC file into the mnemonic format.  For a full list of console commands, enter:
>>/Applications/MarcEdit.app/Contents/MacOS/MarcEdit –help

In the future, the MarcEdit install program will be setting an environmental variable ($MARCEDIT_PATH) on installation.  At this point, I recommend opening your .bash_profile, and add the following line:
export MARCEDIT_PATH=/Applications/MarcEdit.app/Contents/MacOS

You can get this download from: http://marcedit.reeset.net/downloads 

–tr

Jun 052016
 

Over the past month, I’ve been working with ExLibris (thank you to Ori Miller at ExLibris) and Boston College (thanks to Margaret Wolfe) to provide direct integration between MarcEdit and Alma via the Alma Apis.  Presently, the integration allows users to search, create, and update records.  Setup is pretty easy (I think) and once you have your API access setup correctly – you should be off and running.  But, it will be interesting to see if that’s the case as more people play around with this in their sandboxes.

Setting up integration

MarcEdit Alma integration requires that you configure an API key with Alma that supports the bib api and the user api.  The bib api represents the endpoints where the record editing and retrieval happen, while the user api is used to provide a thin layer of authentication before MarcEdit attempts to run an operation (since Alma doesn’t have it’s own authentication process separate from having a key). 

I’d recommend testing this first in your Sandbox.  To do this, you’ll need to know your sandbox domain, and be able to configure the API accordingly.  If you don’t know how to do this, you’ll want to contact ExLibris. 

Once you have your API key, open MarcEdit’s main window and click the Preferences icon.

image

This will open the Preference’s window.  Select the ILS Integration Link, and then check the Enable ILS Integration Checkbox, select Alma from the listbox and then enter the domain for your sandbox.  Alma’s API doesn’t require a username, so leave that blank, but enter your API key into the Password Textbox.  Finally, you’ll need to have setup a Z39.50 connection to your instance.  This is how MarcEdit searches Alma for record retrieval.  If you haven’t setup a Z39.50 Connection, you can do that here, or you can open the Z39.50 Client, Select Modify Databases, Add a new Z39.50 Server, and enter the information for your Alma Instance.  Here’s an example configuration (minus the username and password) for Boston College’s Sandbox:

image

With your Z39.50 Server configured and selected – the ILS Integration Preference’s window will look something like this:

image

Save these settings.  Now, when you open the MarcEditor, you’ll see a new menu item:

image

This menu item will allow you to search and update/create records.  To find items, click on the menu and select Search.  You’ll get the following window:

image

If I run a search for Boston, I’ll retrieve 5 results based on the limit set in the Limit textbox:

image

You can either download all the items by clicking the Download All Items, or you can select the items individually that you want to download, and right click on the Results.  This will give you a menu allowing you to download the records. 

When downloaded, the record will be opened into MarcEdit like the below:

image

Couple notes about the download.  If the download includes an 852 (and they can) – you’ll want to delete that field, otherwise the field will get duplicated.  Right now, I’m trying to figure out if MarcEdit should just remove the value, or if there is an applicable use case for keeping it. 

Download the record, make the edits that you want to make to the record, and then click the Update/Create option from the Alma window.

image

When you click the Update/Create – the tool will upload your data to your Alma server.  If there is an error, you’ll receive the returned error message.  If the process was successful, you’ll get an message telling you that the data had been processed. If you are interesting in seeing the resulting XML output – MarcEdit automatically copies the data to the clipboard. 

Couple of notes about the process – in my testing, I found that updating Serials records was spotty.  I’m thinking this might have something to do with permissions – but I’m not positive about that.  I’m hoping to do a bit more investigation – but I wanted to get this out for folks to start playing with it and maybe providing some feedback.

Secondly, there is a holdings API – it would be possible to allow users to modify holdings data via MarcEdit, but I’d need use-cases in order to see how it fits into this process.

I’m sure this will be a process that I’ll be refining over the next few weeks – but in the mean time, I’d welcome any and all comments. 

–tr

* I’ll be posting a short youtube video and will update the url here.

Apr 112015
 

This all started with a conversation over twitter (https://twitter.com/_whitni/status/583603374320410626) about a week ago.  A discussion about why the current version of MarcEdit is so fragile when being run on a Mac.  The short answer has been that MarcEdit utilizes a cross platform toolset when building the UI which works well on Linux and Windows systems, but tends to be less refined on Mac systems.  I’ve known this for a while, but to really do it right, I’d need to develop a version of MarcEdit that uses native Mac APIs, which would mean building a new version of MarcEdit for the Mac (at least, the UI components).  And I’ve considered it – mapped out a road map – but what’s constantly stopped me has been a lack of interest from the MarcEdit community and a lack of a Mac system.  On the community-side, I can count on two hands the number of times I’ve had someone request a version of MarcEdit  specifically for a Mac.  And since I’ve been making a Mac App version of MarcEdit available – it’s use has been fairly low (though this could be due to the struggles noted above).  With an active community of over 20,000, I try to put my time where it will make the most impact, and up until a week ago, better support for Mac systems didn’t seem to be high on the list.  The second reason is I don’t own a Mac.  My technology stack is made up of about a dozen Windows and Linux systems embedded around my house because they play surprisingly well together, where as, Apple’s walled garden just doesn’t thrive within my ecosystem.  So, I’ve been waiting and hoping that the cross-platform toolset would get better and that in time, this problem would eventually go away.

I’m giving that background because apparently I’ve been misreading the MarcEdit community.  As I said, this all started with this conversation on twitter (https://twitter.com/_whitni/status/583603374320410626) – and out of that, two co-conspirators, Whitni Watkins and Francis Kayiwa set out to see just how much interest there actually was in having dedicated version of MarcEdit for the Mac.  The two set out to see if they could raise funds to acquire a Mac to do this development and indirectly, demonstrate that there was actually a much larger slice of the community interested in seeing this work done.  And, so, off they went – and I set back and watched.  I made a conscious decision that if this was going to happen, it was going to be because the community wanted it and in that, my voice wasn’t necessary.  And after 8 days, it’s done.  In all, 40 individuals contributed to the campaign, but more importantly to me, I heard directly from around 200+ individuals that were hopeful that this project would proceed. 

Development Roadmap

Now the hard work starts.  MarcEdit is a program that has been under constant development since 1999 – so even just rewriting the UI components of the application will be a significant undertaking.  So, I’m breaking up this work in chunks.  I figure it would take approximately 8-12 months to completely port the UI, which is a long-time.  Too long…so I’m breaking the development into 3 month “sprints”.  the first sprint will target the 80%, the functionality that would make MarcEdit productive when doing MARC editing.  This means porting the functionality for all the resources found in the MARC Tools and much of the functionality found in the MarcEditor components.  My guess is these two components are the most important functional areas for catalogers – so finishing those would allow the tool to be immediately useful for doing production cataloging and editing.  After that – I’ll be able to evaluate the remainder of the program and begin working on functional parity between all versions of the application. 

But I’ll admit, at this point, the road map is somewhat even cloudy to me.  See, I’ve written up the following document (http://1drv.ms/1ake4gO) and shared it with Whitni and asked her to work with other Mac users to refine the list and let me know what falls into that 80%.  So, I’ll be interested to see where their list differs from my own.  In the mean time, I’ll be starting work on the port – creating wireframes and spending time over the next week hitting the books and familiarizing myself with Apple’s API docs and the UI best practices (though, I will be trying to keep the program looking very familiar to the current application – best practices be damned).  Coding on the new UI will start in earnest around May 1 – and by August 1, 2015, I hope to have the first version built specifically for a Mac available.  For those interested in following the development process – I’ll be creating a build page on the MarcEdit website (http://marcedit.reeset.net) and will be posting regular builds as new areas of the application are ported so that folks can try them, and give feedback. 

So, that’s where this stands and this point.  For those interested in providing feedback, feel free to contact me directly at reeset@gmail.com.  And for those of you that reached out or participated in the campaign to make this happen, my sincere thanks. 

–TR

MarcEdit 101 Webinar Series

 MarcEdit  Comments Off on MarcEdit 101 Webinar Series
Mar 312015
 

The MarcEdit 101 Webinar Series were created over the course of multiple months for the CARLI (http://www.carli.illinois.edu/) consortium in Spring 2015.  In late March 2015, CARLI reached out to me and requested that these webinars be made available to the larger MarcEdit community, so if you find these webinars useful, please reach out and thank the folks at CARLI.

Couple of notes, these webinars are being made available as is, save for the following modifications:

  1. Attendee names have been anonymized.  While I’m certain most attendees would have no problem with their names showing up in these webinar lists, the original intended audience was locally scoped to CARLI and it’s members.  Masking attendees was done primarily because of this change of scope.
  2. The Q/A at the end of the sessions has generally been removed from the webinars.  Again, these are localized webinars and questions asked during the webinars tend to be within the scope of this consortia.

I’ll be making these video available over the next couple of months.  Again, if you find these webinars useful, please make sure you let the folks at CARLI know.

Series URL: http://marcedit.reeset.net/marcedit-101-workshop

–TR

Dec 192014
 

Over the past couple of weeks, I’ve been working on expanding the linking services that MarcEdit can work with in order to create identifiers for controlled terms and headings.  One of the services that I’ve been experimenting with is NLM’s beta SPARQL endpoint for MESH headings.  MESH has always been something that is a bit foreign to me.  While I had been a cataloger in my past, my primary area of expertise was with geographic materials (analog and digital), as well as traditional monographic data.  While MESH looks like LCSH, it’s quite different as well.  So, I’ve been spending some time trying to learn a little more about it, while working on a process to consistently query the endpoint to retrieve the identifier for a preferred Term. Its been a process that’s been enlightening, but also one that has led me to think about how I might create a process that could be used beyond this simple use-case, and potentially provide MarcEdit with an RDF engine that could be utilized down the road to make it easier to query, create, and update graphs.

Since MarcEdit is written in .NET, this meant looking to see what components currently exist that provide the type of RDF functionality that I may be needing down the road.  Fortunately, a number of components exist, the one I’m utilizing in MarcEdit is dotnetrdf (https://bitbucket.org/dotnetrdf/dotnetrdf/wiki/browse/).  The component provides a robust set of functionality that supports everything I want to do now, and should want to do later.

With a tool kit found, I spent some time integrating it into MarcEdit, which is never a small task.  However, the outcome will be a couple of new features to start testing out the toolkit and start providing users with the ability to become more familiar with a key semantic web technology,  SPARQL.  The first new feature will be the integration of MESH as a known vocabulary that will now be queried and controlled when run through the linked data tool.  The second new feature is a SPARQL Browser.  The idea here is to give folks a tool to explore SPARQL endpoints and retrieve the data in different formats.  The proof of concept supports XML, RDFXML, HTML. CSV, Turtle, NTriple, and JSON as output formats.  This means that users can query any SPARQL endpoint and retrieve data back.  In the current proof of concept, I haven’t added the ability to save the output – but I likely will prior to releasing the Christmas MarcEdit update.

Proof of Concept

While this is still somewhat conceptual, the current SPARQL Browser looks like the following:

image

At present, the Browser assumes that data resides at a remote endpoint, but I’ll likely include the ability to load local RDF, JSON, or Turtle data and provide the ability to query that data as a local endpoint.  Anyway, right now, the Browser takes a URL to the SPARQL Endpoint, and then the query.  The user can then select the format that the result set should be outputted.

Using NLM as an example, say a user wanted to query for the specific term: Congenital Abnormalities – utilizing the current proof of concept, the user would enter the following data:

SPARQL Endpoint: http://id.nlm.nih.gov/mesh/sparql

SPARQL Query:

PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
PREFIX owl: <http://www.w3.org/2002/07/owl#>
PREFIX meshv: <http://id.nlm.nih.gov/mesh/vocab#>
PREFIX mesh: <http://id.nlm.nih.gov/mesh/>

SELECT distinct ?d ?dLabel 
FROM <http://id.nlm.nih.gov/mesh2014>
WHERE {
  ?d meshv:preferredConcept ?q .
  ?q rdfs:label 'Congenital Abnormalities' . 
  ?d rdfs:label ?dLabel . 
} 
ORDER BY ?dLabel 

Running this query within the SPARQL Browser produces a resultset that is formatted internally into a Graph for output purposes.

image

image

image

The images snapshot a couple of the different output formats.  For example, the full JSON output is the following:

{
  "head": {
    "vars": [
      "d",
      "dLabel"
    ]
  },
  "results": {
    "bindings": [
      {
        "d": {
          "type": "uri",
          "value": "http://id.nlm.nih.gov/mesh/D000013"
        },
        "dLabel": {
          "type": "literal",
          "value": "Congenital Abnormalities"
        }
      }
    ]
  }
}

The idea behind creating this as a general purpose tool, is that in theory, this should work for any SPARQL endpoint.   For example, the Project Gutenberg Metadata endpoint.  The same type of exploration can be done, utilizing the Browser.

image

Future Work

At this point, the SPARQL Browser represents a proof of concept tool, but one that I will make available as part of the MARCNext research toolset:

image

As part of the next update.  Going forward, I will likely refine the Browser based on feedback, but more importantly, start looking at how the new RDF toolkit might allow for the development of dynamic form generation for editing RDF/BibFrame data…at least somewhere down the road.

–TR

[1] SPARQL (W3C): http://www.w3.org/TR/rdf-sparql-query/
[2] SPARQL (Wikipedia): http://en.wikipedia.org/wiki/SPARQL
[3] SPARQL Endpoints: http://www.w3.org/wiki/SparqlEndpoints
[4] MarcEdit: http://marcedit.reeset.net
[5] MARCNext: http://blog.reeset.net/archives/1359

MarcEdit Start Page Changes

 MarcEdit  Comments Off on MarcEdit Start Page Changes
Aug 162013
 

**A number of members of the MarcEdit community provided feedback while working on these changes.  Specifically, Heidi Frank (NYU) and Jim Taylor (of http://jtdata.com) for contributing their time and some artistic skill in creating some of the new functional icons.

In addition to a handful of bug fixes and enhancements, one of the big changes coming to the next MarcEdit update will be around UI changes.  I’ve been taking some time and collapsing menus to try and shorten them a bit (they are getting long) and refreshing a few of the screens and tools.  The first screen to be refreshed and includes some significant enhancements is the start screen.

Current Start Screen

The MarcEdit start screen has been largely unchanged for close to 5 years.  The start screen has included a start screen that includes access to a handful of tools and utilities that I have heard are fairly commonly used. 

marcedit5starup 
Current MarcEdit Start Screen

Over the years, I periodically change the tools and utilities available on the start page, but by and large, it has stayed largely static. 

Updated Start Screen

The next update will reflect a shift in the start screen design.  First, the page will move from a more textual/information screen, to one that is more reliant on both graphics and text to help users find the right tool.  Secondly, the start screen will be customizable.  The screen will provide the ability for users to define what tools that they want to have quick access too. 

image 
Updated Default Screen

The Updated Start Screen will include larger images, with text – to help users quickly locate the tool that they are looking for on the start screen.  However, unlink past versions, users can change the tools available from this screen.  By clicking on the lower right hand configurations icon, or selecting Preferences from the Tools menu, users will be presented with the following new configuration option:

image
New Configuration Options

The next configuration options pull out the 12 most commonly used tools/add-ins.  Users can select up to 4 of these items and place them on their start screen.  By selecting new items, and clicking OK, the user will find that their application lay out changes:

image 
User Configured Interface

Here, I changed the default options to selected the Delimited Text Translator, the Merge Records Tool, the RDA Helper, and the Call Number Generator.  These will now be available to me on the front screen whenever I open MarcEdit.  And since these configuration changes are linked to a user’s profile, multiple users, on the same computer, could have different Start Screens depending on how the utilize the program.

image 
Selecting 3 items

As noted above, you can select up to 4 user tools to display on the front page.  But users have the option to select as few options as they want as well.  In this example, I removed an option and only selected the most common 3 tools for the Start Screen.

 

Conclusion

These UI changes are the first of what will be a handful of changes that I’ll be making to the tool over the next couple of months as I refresh the interface, clean up some old code and look to improve some of the workflows in the application.  I’ll be posting wireframes through the MarcEdit listserv when I’m planning major changes, so if you are interested in having a voice on upcoming changes, keep and eye open on the MarcEdit list.

–tr

MarcEdit 5.8 Delimited Text Translator – Auto Generate Arguments List

 MarcEdit  Comments Off on MarcEdit 5.8 Delimited Text Translator – Auto Generate Arguments List
Nov 222012
 

One of the often requested enhancements to the MarcEdit Delimited Text Translator is the ability to auto generate the arguments list.  For many users, their spreadsheets or delimited text documents include a line at the beginning of the document defining the data found in the file.  I’ve often had folks wonder if I could do anything with that data to help auto generate the arguments list used by MarcEdit to translate the data. 

Well, in anticipation of Thanksgiving, I finished working on what will be the next MarcEdit update.  I won’t post it till the weekend, but this new version will include and Arguments Auto Generation button that will allow MarcEdit to capture the first line of a data file and if properly formatted, auto configure the Arguments List. 

Format:

The format supported by the Auto Generation feature is pretty straightforward.  It essentially is the following:  Field$Subfield[ind1ind2punct].

Let me break down the format definition:

  • Field – represents the field to be mapped to, i.e.: 245.  This is a required value.
  • $Subfield – represents the subfield to be mapped., i.e.: $a.  This is a required value.
  • ind1 – represents the first indicator.  This is an optional value, but if defined, indicator 2 must be defined.
  • ind2 – represents the second indicator.  This is an optional value, but if indicator 1 is defined, indicator 2 must also be defined.
  • punct – represents the trailing punctuation of the field.  This is an optional value.  However, if you wish to define the punctuation, you must define the indicator 1 and indicator 2 values as well.

Some examples of the syntax:

  • 245$a  — no indicators are defined, the default indicators, 2 blanks, will be used.
  • 245$a10 – defines the field, subfield and indicators 1 and 2.
  • 245$a10. – defines the field, subfield, indicators 1 and 2, and defines a period as the trailing punctuation.

In MarcEdit, you can join fields together.  This allows users to join data in multiple columns into a single subfield.  In MarcEdit, joined fields are represented by an asterisk “*”.  If I wanted to join two or more fields, I can add an asterisk group to the field.  For example:

  • Field0:  *100$a10,
  • Field1:  *100$a10.

MarcEdit will interpret field 0 and field 1 as being joined fields because the asterisk marks them as joined.

I’ve placed a video on YouTube to demonstrate the upcoming functionality.  You can find out more about it here:

If you have questions about this new function or suggestions, let me know.

–TR

MarcEdit 5.2 Update

 MarcEdit  Comments Off on MarcEdit 5.2 Update
Mar 082010
 

Hi all,

I have just uploaded a new version of MarcEdit to the website.  This version specifically addresses two bugs and introduces the Task Automation function into the application. 

Bug Fixes:

  • Changes not saved in the MarcEditor:
    Under certain conditions, MarcEdit would lose track of changes made within the program.  This occurred primarily after changes had been made and then the Find All Function was used.  This has been corrected as of this version.
  • Save As…file not found error:
    When using Save As to Save data, MarcEdit would throw an error if the file being saved did not previously exist.  This has been corrected as of this version.
  • Invalid prompts to save a file before closing:
    MarcEdit tended to error on the side of always asking users to save their data before closing.   However, even if a user had saved their data, the message may still have popped up.  This has been corrected as of this version.
  • Find/Replace – replacing without using Match Case:
    While it is always recommended when doing global replacements to do them using the Match Case option – prior to this version, unchecking the match case option would cause the replacement to take too many characters.  This has been corrected as of this version.

Enhancements:

  • Task Automation tool:
    The task automation tool is a recorder that allows users chain together replacement functions.  Sadly, I haven’t been able to add information to the help file, however, I did record the following video tutorial to provide some initial background to get started using the function.  One quick note – this is a new tool so when using it, please verify changes.  Moreover, feedback is definitely appreciated.
    Video Tutorial:   http://www.youtube.com/watch?v=gmqTGfTubU4

You can download the updated version of MarcEdit at: MarcEdit_Setup.msi or the Linux/Mac/Other version build at: marcedit.zip.

–TR

Aug 042009
 

I was asked the other day at AALL (American Association of Law Libraries) if MarcEdit could be used to move specific data from one field and replace the data currently present in another.  So, an example – the ability to move data from a 260$c to the 008 position 7:4.  You can actually, though its sadly not documented (one of those few hidden gems that have been created either for specific projects I or others have been working on).  So how do we do it.

Open the Edit Subfield Function.  In the Edit subfield function, there is an option called Move Subfield data.  That’s that one we want to check.   Then, we enter the following (using the 260$c-008).

Field: 260 [Enter the field with the data that you wish to move]
Subfield: c
Find: [leave blank – though you can enter data here if you want to find something specific to move]
Replace: 008|7||

Ok, the Replace looks funny and it is.  There are essentially a handful of options you can set here (4) – I’m going to explain two for now (and will update this post when I update the official documentation). 

Each pipe “|? represents a delimiter.  The first two pipes are the most important:

  1. Field to move to
  2. Where to move (replace) the data

In the above example, we are moving data to the 008 and placing the data in position 7.  If I was placing the data into a subfield, I would have entered a subfield (example: c) here.  So, the edit form would look like the following:

image

 

–TR