Aug 282016
 

One of the gaps in the Mac version of MarcEdit has been the lack of a console mode.  This update should correct that.  However, a couple things about how his works…

1) Mac applications are bundles, so in order to run the console program you need to run against the application bundle.  What does this look like?   From the terminal, one would run
>>/Applications/MarcEdit.app/Contents/MacOS/MarcEdit –console

The –console flag initializes the terminal application and prompts for file names.  You can pass the filenames (this must be fully qualified paths at this point) via command-line arguments rather than running in an interactive mode.  For example:
>>/Applications/MarcEdit.app/Contents/MacOS/MarcEdit –s /users/me/Desktop/source.mrc –d /users/me/Desktop/output.mrk –break

The above would break a MARC file into the mnemonic format.  For a full list of console commands, enter:
>>/Applications/MarcEdit.app/Contents/MacOS/MarcEdit –help

In the future, the MarcEdit install program will be setting an environmental variable ($MARCEDIT_PATH) on installation.  At this point, I recommend opening your .bash_profile, and add the following line:
export MARCEDIT_PATH=/Applications/MarcEdit.app/Contents/MacOS

You can get this download from: http://marcedit.reeset.net/downloads 

–tr

Aug 082016
 

I’ve been putting together some information to help a few non-catalogers understand what MarcEdit is an how much it get’s used around the world.  Couple of fun stats for 2015 based only on usage information from users that make use of the automated update tool.

2015 Usage Information:

  • ~3 million unique program executions (times the program checked for an update [was started])
  • 192 unique countries/political regions
  • ~18,000 unique/active users
  • around 105 print/online citations in 2015 related to MarcEdit (combination of Google Scholar, WOS data – though, this wasn’t rigorously scrubbed for dups)

Where is MarcEdit used?

image

Each of these points (saved the couple in the US where Google got a little confused with Georgia, Jersey, etc.), represents an individual country with users working with MarcEdit.  In 2015, the US represented ~55 percent of the unique usage.  That means that 45% of the MarcEdit community is outside of the United States.

–tr

 Posted by at 11:51 am
Jul 112016
 

The past 3 weeks, I’ve been doing a lot of work on MarcEdit.  These initial changes impact just the windows and linux version of MarcEdit.  I’ll be taking some time tomorrow and Wed. to update the Mac version.  The current changes are as follows:

6.2.190
* Enhancement: Language files have been updated
* Enhancement: Command-line tool: -task option added to support tasks being run via the command-line.
* Enhancement: Command-line tool: -clean and -validate options updated to support structure validation.
* Enhancement: Alma integration: Updating version numbers and cleaned up some windowing in the initial release.
* Enhancement: Small update to the validation rules file.
* Enhancement: Update to the linked data rules file around music headings processing.
* Enhancement: Linked Data Platform: collections information has been moved into the configuration file.  This will allow local indexes to be added so long as they support a json return.
* Enhancement: Merge Records — 001 matching now looks at the 035 and included oclc numbers by default.
* Enhancement: MarcEngine: Updated the engine to accommodate invalid data in the ldr.
* Enhancement: MARC SQL Explorer — added an option to allow mysql database to be created as UTF8.
* Enhancement: Handful of odd UI changes.

You can get the update from the downloads page (http://marcedit.reeset.net/downloads) or via the automated update tools.

–tr

 Posted by at 8:39 pm
Jul 112016
 

MarcEdit’s command-line function has always had the ability to run validation tasks against the MarcEdit rules file.  However, the program hasn’t included access to the cleaning functions of the validator.  As of the last update, this has changed.  If the –validate command is invoked without a rules file defined, the program will validate the structure of the data.  If the –clean option is passed, the program will remove invalid structural data from the file. 

Here’s an example of the command:

>> cmarcedit.exe -s “C:\Users\rees\Desktop\CLA_UCB 2016\Data File\sample data\bad_sample_records.mrc” –validate

–tr

 Posted by at 8:19 pm
Jul 112016
 

MarcEdit’s task list functionality has made doing repetitive tasks in MarcEdit a fairly simple process.  But one limitation has always been that the tasks must be run from within MarcEdit.  Well, that limitation has been lifted.  As of the last update, a new option has been added to the command-line tool: –task.  When run with a path to a task to run, MarcEdit will preform the task from the command-line. 

Here’s an example of a command:

cmarcedit.exe -s “C:\Users\rees\Desktop\withcallnumbers.mrk” -d “C:\Users\rees\Desktop\remote_task.mrk” -task “C:\Users\rees\AppData\Roaming\marcedit\macros\tasksfile-2016_06_17_190213223.txt”

This functionality is only available in the Windows and Linux version of the application.

–tr

 Posted by at 8:15 pm
Jul 072016
 

Over the past year and half, I’ve been working with MarcEdit to provide the ability for users interested in doing URI entification against various known vocabularies, as well as adding these vocabularies into an authority processing workflow.  The processes work well for the handful (15-20) defined indexes that MarcEdit currently queries.  But long-term, libraries may see a proliferation of indexing services, and the ability to add custom linking services into MarcEdit will become more important. 

So, with the next version of MarcEdit – I’ll be making a change.  While MarcEdit will still have custom code that handles some of the larger national indexes (because this makes sense) – all the information about the collections, their internal labels, and JSON object paths (for values) will be moved into the rules file.  The upside to this is that users will be able to customize the queries MarcEdit is making against a linked data service (if they want) – but more important, they can add their own linked data services.  Initially, custom services will need to return well-formed JSONLD, where the value for the specified URI can be referenced as a JSON object path (i.e., this the path used with the AAT index: results.bindings[0].subj.value), but if it matches that criteria, you’ll be able to add your own services into MarcEdit, and if MarcEdit encounters your defined index label within your metadata, it will ping the service. 

Within the rules file, you’ll see this new structure represented as:

<collection>
      <name>US Library of Congress Subject Heading</name>
      <label>lcsh</label>
      <uri>http://id.loc.gov/authorities/subjects/label/{search_terms}</uri>
      <path />
    </collection>

Since many of the indexes (national indexes) MarcEdit queries now do not use JSON in their return (or MarcEdit utilizes a special, optimized process) – these will not have paths.  But for other sources like MESH headings, Getty headings, you’ll be able to see how this structure is being utilized.

I’m still testing these changes – as the changes to the linked data framework affect a number of parts of the application – but these will be part of the next update.

–tr

 Posted by at 11:17 am

MarcEdit Update

 MarcEdit  Comments Off on MarcEdit Update
Jun 152016
 

Last night, I posted an update squashing a couple bugs and adding some new features.  Here’s the change log:

* Bug Fix: Merge Records Tool: If the user defined field is a title, the merge doesn’t process correctly.
* Bug Fix: Z39.50 Batch Processing: If the source server provides data in UTF8, characters from multi-byte languages may be flattened.
* Bug Fix: ILS Integration..Local:  In the previous version, one of the libraries versions didn’t get updated and early beta testers had some trouble.
* Enhancement: Join Records — option added to process subdirectories.
* Enhancement: Batch Processing Tool — option added to process subdirectories
* Enhancement: Extract Selected Records — Allowing regular expressions as an option when processing file data.
* Enhancement: Alma Integration UI Improvements

Downloads can be picked up via the automated updating tool or via the downloads (http://marcedit.reeset.net/downloads) page.

 

–tr

 Posted by at 2:36 pm

MarcEdit Updates

 Uncategorized  Comments Off on MarcEdit Updates
Jun 052016
 

This update has been a little while coming and represents a significant number of updates, bug fixes and enhancements.  On the Mac side, the two largest updates where the implementation of the Delimited Text Translator and the OAI Harvester, on all platforms – Alma Integration.  You can find notes about the Alma integration here: http://blog.reeset.net/archives/1950

Please see the full list of updates below.  Downloads can be picked up through the automatic update mechanism in MarcEdit or via the downloads page at: http://marcedit.reeset.net/downloads

–tr

MarcEdit Windows/Linux Updates:

* Bug Fix: ILS Integration: Local Integration — corrected display rendering and search for keyword
* Bug Fix: Add/Delete Records — Corrected problem when using the Add field only if not a duplicate option
* Enhancement: Validate Headings — added dynamic caching
* Enhancement: Build Links — added dynamic caching
* Enhancement: ILS Integration — First version of Alma integration
* Bug Fix: Math conversion — Degree/minute/seconds to Degrees correction
* Settings Change: Updated the RDA Field conversion to limit abbreviation checking in the 245 field to the 245$c
* Enhancement: RDA Abbreviations — new abbreviations added
* Enhancement: Select/Delete MARC Records — added option to expose specialized search options like Field # searching, Range Searching, File Searching and Size searching.
* Bug Fix: OAI Harvester — Debug URL wasn’t correct when adding date values.
* Bug Fix: RDA Helper — Added Data validation to ensure that invalid 008 data doesn’t cause a data crash.
* Enhancement: Delimited Text Translator — Added more preview options
* Enhancement: Delimited Text Translator — Added Holdings LDR/008 values
* Enhancement: UI Improvements — a large number of textboxes that accept file paths now support drag and drop.

MarcEdit Mac Updates:

* Bug Fix: ILS Integration: Local Integration — corrected display rendering and search for keyword
* Bug Fix: Add/Delete Records — Corrected problem when using the Add field only if not a duplicate option
* Enhancement: Validate Headings — added dynamic caching
* Enhancement: Build Links — added dynamic caching
* Enhancement: ILS Integration — First version of Alma integration
* Bug Fix: Math conversion — Degree/minute/seconds to Degrees correction
* Settings Change: Updated the RDA Field conversion to limit abbreviation checking in the 245 field to the 245$c
* Enhancement: RDA Abbreviations — new abbreviations added
* Enhancement: Select/Delete MARC Records — added option to expose specialized search options like Field # searching, Range Searching, File Searching and Size searching.
* Bug Fix: RDA Helper — Added Data validation to ensure that invalid 008 data doesn’t cause a data crash.
* Enhancement: UI Improvements — a large number of textboxes that accept file paths now support drag and drop.
* Enhancement: OAI Harvester Implemented
* Enhancement: Delimited Text Translator implemented

 Posted by at 10:27 pm
Jun 052016
 

Over the past month, I’ve been working with ExLibris (thank you to Ori Miller at ExLibris) and Boston College (thanks to Margaret Wolfe) to provide direct integration between MarcEdit and Alma via the Alma Apis.  Presently, the integration allows users to search, create, and update records.  Setup is pretty easy (I think) and once you have your API access setup correctly – you should be off and running.  But, it will be interesting to see if that’s the case as more people play around with this in their sandboxes.

Setting up integration

MarcEdit Alma integration requires that you configure an API key with Alma that supports the bib api and the user api.  The bib api represents the endpoints where the record editing and retrieval happen, while the user api is used to provide a thin layer of authentication before MarcEdit attempts to run an operation (since Alma doesn’t have it’s own authentication process separate from having a key). 

I’d recommend testing this first in your Sandbox.  To do this, you’ll need to know your sandbox domain, and be able to configure the API accordingly.  If you don’t know how to do this, you’ll want to contact ExLibris. 

Once you have your API key, open MarcEdit’s main window and click the Preferences icon.

image

This will open the Preference’s window.  Select the ILS Integration Link, and then check the Enable ILS Integration Checkbox, select Alma from the listbox and then enter the domain for your sandbox.  Alma’s API doesn’t require a username, so leave that blank, but enter your API key into the Password Textbox.  Finally, you’ll need to have setup a Z39.50 connection to your instance.  This is how MarcEdit searches Alma for record retrieval.  If you haven’t setup a Z39.50 Connection, you can do that here, or you can open the Z39.50 Client, Select Modify Databases, Add a new Z39.50 Server, and enter the information for your Alma Instance.  Here’s an example configuration (minus the username and password) for Boston College’s Sandbox:

image

With your Z39.50 Server configured and selected – the ILS Integration Preference’s window will look something like this:

image

Save these settings.  Now, when you open the MarcEditor, you’ll see a new menu item:

image

This menu item will allow you to search and update/create records.  To find items, click on the menu and select Search.  You’ll get the following window:

image

If I run a search for Boston, I’ll retrieve 5 results based on the limit set in the Limit textbox:

image

You can either download all the items by clicking the Download All Items, or you can select the items individually that you want to download, and right click on the Results.  This will give you a menu allowing you to download the records. 

When downloaded, the record will be opened into MarcEdit like the below:

image

Couple notes about the download.  If the download includes an 852 (and they can) – you’ll want to delete that field, otherwise the field will get duplicated.  Right now, I’m trying to figure out if MarcEdit should just remove the value, or if there is an applicable use case for keeping it. 

Download the record, make the edits that you want to make to the record, and then click the Update/Create option from the Alma window.

image

When you click the Update/Create – the tool will upload your data to your Alma server.  If there is an error, you’ll receive the returned error message.  If the process was successful, you’ll get an message telling you that the data had been processed. If you are interesting in seeing the resulting XML output – MarcEdit automatically copies the data to the clipboard. 

Couple of notes about the process – in my testing, I found that updating Serials records was spotty.  I’m thinking this might have something to do with permissions – but I’m not positive about that.  I’m hoping to do a bit more investigation – but I wanted to get this out for folks to start playing with it and maybe providing some feedback.

Secondly, there is a holdings API – it would be possible to allow users to modify holdings data via MarcEdit, but I’d need use-cases in order to see how it fits into this process.

I’m sure this will be a process that I’ll be refining over the next few weeks – but in the mean time, I’d welcome any and all comments. 

–tr

* I’ll be posting a short youtube video and will update the url here.

MarcEdit Update

 MarcEdit  Comments Off on MarcEdit Update
May 232016
 

Yesterday, I posted a significant update to the Windows/Linux builds and a maintenance update to the Mac build that includes a lot of prep work to get it ready to roll in a number of changes that I’ll hopefully complete this week.  Unfortunately, I’ve been doing a lot of travelling, which means that my access to my mac setup has been pretty limited and I didn’t want to take another week getting everything synched together. 

So what are the specific changes:

ILS Integrations
I’ve been spending a lot of time over the past three works head down working on ILS integrations.  Right now, I’m managing two ILS integration scenarios – one is with Alma and their API.  I’m probably 80% finished with that work.  Right now, all the code is written, I’m just not getting back expected responses from their bibliographic update API.  Once I sort out that issue – I’ll be integrating this change into MarcEdit and will provide a youtube video demonstrating the functionality. 

The other ILS integration that I’ve been accommodating is working with MarcEdit’s MARC SQL Explorer and the internal database structure.  This work builds on some work being done with the Validate Headings tool to close the authority control loop.  I’ll likely be posting more about that later this week as I’m currently have a couple libraries test this functionality to make sure I’ve not missed anything.  Once they give me the thumbs up, this will make its way into the MarcEditor as well. 

But as part of this work, I needed to create a way for users to edit and search the local database structure in a more friendly way.  So, leveraging the ILS platform, I’ve included the ability for users to work with the local database format directly within the MarcEditor.  You can see how this works here (https://www.youtube.com/watch?v=dMJ_pUxyoFc&feature=youtu.be): Integrating the MarcEditor with a local SQL store.  I’m not sure what the ideal use case is for this functionality – but over the past couple of weeks, it had been requested by a couple of power users currently using the MARC SQL Explorer for some data edits, but hoping for an easier to user interface.  This work will be integrated into the Mac MarcEdit version at the end of this week.  All the prep work (window/control development) has been completed.  At this point, its just migrating the code so that it works within the Mac’s object-C codebase.

Edit Shortcuts
I created two new edit shortcuts in the MarcEditor.  The first, Find Records With Duplicate Tags, was created to help users look for records that may have multiple tags or a tag/subfield combination with a set of records.  This is work that can be done in the Extract Selected Records tool, but it requires a bit a trickery and knowledge of how MarcEdit formats data. 

image

How does this work – say you wanted to know which records had multiple call numbers (050) fields in a record.  You would select this option, enter 050 in the prompt, and then the tool would create for you a jump list showing all the records that met your criteria. 

Convert To Decimal Degrees
The second Edit ShortCut function is the first Math function (I’ll be adding two more, specifically around finding records with dates greater than or less than a specific value) targeting the conversion of Degree/Minutes/Seconds to decimal degrees.  The process has been created to be MARC agnostic, so users can specify the field, and subfields to process.  To run this function, select it from the Edit Shortcuts as demonstrated in the screenshot below:

image

When selected, you will get the following prompt:

image

This documents the format for defining the field/subfields to be processed.  Please note, it is important to define the all four potential values for conversion – even if they are not used within the record set. 

Using this function, you can now convert a value like:
=034  1\$aa$b1450000$dW1250000$eW1163500$fN0461500$gN0420000
To:
=034  1\$aa$b1450000$d+125.0000$e+116.5800$f+046.2500$g+042.0000

This function should allow users to transition their cartographic data to a format that is much more friendly to geographic interpretation if desired.

Bug Fixes:
This update also addressed a bug in the Build New field parser.  If you have multiple arguments, side-by-side, within the same field grouping (i.e., {100$a}{100$b}{100$c} – the parser can become confused.  This has been corrected.

Updates:
Included and update to the linked data rules file, updating the 7xx fields to include the $t in the processing.  Also updated the UNIMARC translation to include a 1:1 translation for 9xx data.

Over the next week, I hope to complete the Alma integration, but will focusing the development work in my free time on getting the Mac version synched with these changes.

–tr

 Posted by at 6:39 pm