Jul 052015
 

It’s with a little trepidation that I’m formally making the first Public Preview of the MarcEdit OSX version available for download and use.  In fact, as of today, this version is now *the* OSX download available on the downloads page.  I will no longer be building the old code-base for use on OSX.

When I first started this project around Mid-April, I began knowing that this process would take some time.  I’ve been working on MarcEdit continuously for a little over 16 years.  It’s gone through one significant rewrite (when the program moved from Assembly to C#) and has had way too many revisions to count.  In agreeing to take on the porting work — I’d hoped that I could port a significant portion of the program over the course of about 8 months and that by the end of August, I could produce a version of MarcEdit that would cover the 80% or so of the commonly used application toolset.  To do this, it meant porting the MARC Tools portion of the application and the MarcEditor.

Well, I’m ahead of schedule.  Since about 2014, I’ve been reworking a good deal of the application to support a smoother porting process sometime in the future — though, honestly, I wasn’t sure that I’d ever actual do the porting work.  Pleasantly, this early work has made a good deal of the porting work easier allowing me to move faster than I’d anticipated.  As of this posting, a significant portion of that 80% has been converted, and I think that for many people — most of what they probably use daily — has been implemented.  And while I’m ahead of schedule and have been happy with how the porting process has gone, make no mistake — it’s been a lot of work, and a lot of code.  Even though this work has primarily been centered around rewriting just the UI portions of MarcEdit, you are still talking, as of today, close to 200,000 lines of code.  This doesn’t include the significant amount of work I’ve done around the general assemblies that have provided improvements to all MarcEdit users.  Because of that — I need to start getting feedback from users.  While the general assemblies go through an automated testing process — I haven’t, as of yet, come up with an automated testing process for the OSX build.  This means that I’m testing things manually, and simply cannot go through the same leveling of testing that I do each time I build the Windows version.  Most folks may not realize it, but it takes about a day to build the Windows version — as the program goes through various unit tests processing close to 25 million records.  I simply don’t have an equivalent of that process yet, so I’m hoping that everyone interested in this work will give it a spin, use it for real work, and let me know if/when things fall down.

In creating the Preview, I’ve tried to make the process for users as easy as possible.  Users interested in running the program simply need to be running at least OSX 10.8 and download the dmg found here: http://marcedit.reeset.net/downloads.  Once downloaded, run the dmg an a new disk will mount called MarcEdit OSX.  Run this file, and you’ll see the following installer:

MarcEdit OSX installer

MarcEdit OSX installer

Drag the MarcEdit icon into the Applications folder and the application will either install, or overwrite an existing version.  That’s it.  No other downloads are necessary.  On first run, the program will generate a marcedit folder under /users/[yourid]/marcedit.  I realize that this isn’t completely normal — but I need the data accessible outside of the normal app sandbox to easily support updates.  I’d also considered the User Documents folder, but the configuration data probably shouldn’t live there either.  So, this is where I ended up putting it.

So what’s been completed — Essentially, all the MARC Tools functions and a significant amount of the MarcEditor has been completed.  There are some conspicuous functions that are absent at this point though.  The Call Number and Fast Heading generation, the Delimited Text Translator and Exporter, the Select and Delete Selected Records, everything Z39.50 related, as well as the Linked Data tools and the Integration work with OCLC and Koha.  All these are not currently available — but will be worked on.  At this point, what users can do is start letting me know what absent components are impacting you the most, and I’ll see how they fit into the current development roadmap.

Anyway — that’s it.  I’m excited to let you all give this a try, and a little nervous as well.  This has been a significant undertaking which has definitely pushed me a bit, requiring me to learn Object-C in a short period of time, as well as quickly assimilate a significant portion of Apples SDK documents relating to UI design.  I’m sure I’ve missed things, but it’s time to let other folks start working with it.

If you have been interested in this work — download the installer, kick the tires, and give feedback.  Just remember to be gentle.  :)

–TR

Download URL: http://marcedit.reeset.net/downloads

 

 Posted by at 8:40 pm
Jul 052015
 

This was something I’d hoped to get into the last update, but didn’t get the time to test it; so I got it done now.  While at the first MarcEdit User Group meeting at ALA, there was a question about supporting 880 fields when exporting data via tab delimited format.  When you use the tool right now, the program will export all the 880 fields, not a specific 880 field.  This update changes that.  After the update, when you select the 880 field in the Export tab delimited tool, the program will ask you for the linking field.  In this case, the program will then match the 880$6[linkingfield], and pull the selected subfield.  I’m not sure how often this comes up — but it certainly made a lot of sense when the problem was described to me.

You can pick up the download at: http://marcedit.reeset.net/downloads

–tr

 Posted by at 8:33 pm
Jun 302015
 

Changes in this update:

6.1.21
* 6.1.21
** Bug Fix: Conditional Delete - When selecting regular expressions -- there were times when the process wasn't being recognized.
** Enhancement: Conditional Delete - This function use to only work when using the Regular Expression option.  This now works for all options.
** Bug Fix: ValidateISBNs - Process would only process the first subfield.  If the subfield to be processed wasn't the first one, it wouldn't be validated.
** Enhancement: ValidateISSN: Uses mathematical formula to validate ISSNs.
** Bug Fix: Generate Fast Headings (Stand alone tool) -- LDR fields could be deleted.  
** Enhancement: Working to make the global edit functions a little more fault tolerant around record formatting.
** Enhancement: Generate MARC record from URL -- program generates MARC records from Webpages.  If you pass it an LC URL, it will generate data from the MARCXML.

At this point, only the Windows and Linux downloads were updated.  I'll be replacing the Mac download with the first version of the native OSX build the July 4th weekend.  You can get the updates either via the Automated updated tool or from the website at: http://marcedit.reeset.net/downloads

--tr
 Posted by at 8:45 pm
Jun 192015
 

Logistics

Time: 6:00 – 7:30 pm, Friday, June 26, 2015
Place: Marriott Marquis (map)
Room: Pacific H, capacity: 30

Description:

The MarcEdit user community is large and diverse and honestly, I get to meet far too few community members.  This meeting has been put together to give members of the community a chance to come together and talk about the development road map, hear about the work to port MarcEdit to the Mac, and give me an opportunity to hear from the community.  I’ll talk about future work, areas of potential partnership, as well as hearing from you what you’d like to see in the program to make your metadata live’s a little easier.  If this sounds interesting to you — I really hope to see you there.

Acknowledgements:

A *big* thank you to John Chapman and OCLC for allowing this to happen.  As folks might guess, finding space at ALA can be a challenging and expensive endeavor so when I originally broached the idea with OCLC, I had pretty low expectations.  But they truly went above and beyond any reasonable expectation, working with the hotel and ALA so this meeting could take place.  And why they didn’t ask for it — they have my personal thanks and gratitude.  If you can attend the event, or heck, wish you could have but your schedule made it impossible — make sure you let OCLC know that this was appreciated.

 Posted by at 1:24 pm

Working with the Clipboard on OSX

 C#, code, Programming  Comments Off on Working with the Clipboard on OSX
Jun 172015
 

Coming from the Windows and Linux world — the object where data is copy and pasted from is called the Clipboard.  Not so in OSX.  In OSX, this is referred to as the NSPasteBoard.  Should you need to get string data on and off of it – use the following:

 

private static string[] pboardTypes = new string[] { "NSStringPboardType" };
public void SetClipboardText(string text)
{
	NSPasteboard.GeneralPasteboard.DeclareTypes(pboardTypes, null);
	NSPasteboard.GeneralPasteboard.SetStringForType(text, pboardTypes[0]);
}

public string GetClipboardText()
{
	return NSPasteboard.GeneralPasteboard.GetStringForType(pboardTypes[0]);
}

–tr

 Posted by at 3:23 pm

Updating resources on the main UI thread: OSX & C#

 Uncategorized  Comments Off on Updating resources on the main UI thread: OSX & C#
Jun 062015
 

One of the things that has been a learning experience for me has been understanding how OSX handles threading, understanding what objects are thread-safe, and how to capture user feedback from within an thread running outside of the main UI thread.

Conceptually, Windows, OSX, Linux — UIs tend to be run through a single, primary thread.  This is why long-running processes can lock a window from updating.  In Windows, this can be overcome by creating background threads.  While these threads should not interact with the main UI objects — you do have the ability to spawn a new UI thread that can be used to capture interaction from users.  You just need to be careful to ensure that objects stay compartmentalized since these objects are not explicitly thread-safe.

Within the OSX environment, the UI components cannot be run from outside of the main thread.  This means that if you have a background threading process, you cannot interact with the user because all UI objects are marked as not safe for threading.  Not very practical since there are times when user interaction is definitely necessary.  For example, within MarcEdit’s Mac port — I need to be able to tell users when the program has updated — and I’d like this process to occur outside the primary UI thread so not to slow down application startup or user interactions.  To do this, you need to find a way to thread switch — so that you can encapsulate the code that will interact with the UI around the necessary code to switch execution back to the main UI thread.  In C# and on Windows, you can take non-thread-safe components and using a delegate and Invoke, thread switch within your application.  OSX has something similar, but after hours of looking, I wanted to make sure I document how it works here, so I won’t have to look this up again.  The two functions are: InvokeOnMainThread and BeginInvokeOnMainThread.  These two functions have some limitations — they can only be run within the context of a UI thread (so a window controller because you need access to the NSWindow handle) — but from within this block, you have the ability to generate an NSAlert class (though tricky) as well as generate new NSWindow objects to capture user input.

Here’s an example of how this would work below.  This is the code used for MarcEdit’s Mac notification service.  The program starts a thread when the application loads the main window for the first time — that thread initiates a System.Net.HttpWebRequest object to retrieve information about the current MarcEdit Mac Build.  The program then parses the version information, compares it to the local version and build number — and if the remote build number is newer — generates a UI element that tells the user what has changed, and provides buttons to download the new application.  This code lives in a couple of different classes, but these are the relevant parts.

//MainWindowController

public override void AwakeFromNib ()
{
	this.Window.BackgroundColor = NSColor.White;
	base.Window.Center();
	base.AwakeFromNib ();
				
	System.Threading.Thread objT = new System.Threading.Thread((System.Threading.ThreadStart)delegate { CheckUpdate();});
	objT.Start ();

}

void CheckUpdate ()
{
	helpers objH = new helpers ();
	string data = objH.GetWebContent(clsGlobal.MacNotificationURI);
	string sversion = data.Split (System.Environment.NewLine.ToCharArray ()) [0].Trim();

	if (!string.IsNullOrEmpty (data)) {
		NSObject ver = NSBundle.MainBundle.InfoDictionary ["CFBundleVersion"];
		NSObject ver2 = NSBundle.MainBundle.InfoDictionary ["CFBundleShortVersionString"];

		Version vweb = new Version (sversion);
		Version vlocal = new Version (ver2.ToString() + "." + ver.ToString ());

		if (vweb > vlocal) {
			//To generate a UI element from outside the main UI thread, 
			//use InvokeOnMainThread and create a delegate function block
			InvokeOnMainThread (() => {
				frmUpdateNotificationController objU = new frmUpdateNotificationController();
				objU.Window.Center();
				objU.ShowWindow(this);
				objU.LoadMessage("Build #" + sversion + System.Environment.NewLine + System.Environment.NewLine + 
					data);
			});	
                 } 
	}
}

//helpers class

public string GetWebContent(string uri) {

	System.Net.HttpWebRequest myReq =
				(System.Net.HttpWebRequest)System.Net.WebRequest.Create (new Uri (uri));
	System.Net.HttpWebResponse response = (System.Net.HttpWebResponse)myReq.GetResponse ();
	System.IO.Stream responseStream = response.GetResponseStream ();
	System.IO.StreamReader reader = new System.IO.StreamReader (responseStream, System.Text.Encoding.UTF8);
	string val = reader.ReadToEnd ();
	reader.Close ();
	response.Close ();
	return val;
}

 Posted by at 9:28 am

MarcEdit Mac Port Update

 MarcEdit  Comments Off on MarcEdit Mac Port Update
Jun 062015
 

Having made one preview available for evaluation and feedback, I’ve been diligently working on updating the tool and working on new functionality.  This includes moving onto developing a new notification service so that preview users know when new builds are available and providing a new preferences window to enable support for changing applicable preferences currently exposed within the application.  From the perspective of new work — I’ve begun working on the MarcEditor.  At this point, I’ve mocked out the window and am starting to create the global editing toolsets and connecting actions to the various UI elements.  This is going to be a bit of a time consuming process — but thankfully, it’s been made somewhat easier by the fact that much of the code within the MarcEditor that’s not platform specific, has been moved outside the application to re-usable assemblies.  There’s a bit more refactoring that needs to be done, and I’ll need to re-think how the program streams data into the MarcEditor edit window since the OSX apis make this a bit difficult — but it’s getting there.  Below — you will find screenshots of some of the new work?

–tr

Mac Port Notification Example

MarcEdit Mac Port Notifications Window Example

MarcEdit Mac Preferences: MARCEngine

MarcEdit Mac Preferences window: Current preferences exposed for update are for the MARCEngine and the Automatic Update notification.

This is the initial MarcEditor Wireframe for the Mac.  This feels pretty solid at this point.

This is the initial MarcEditor Wireframe for the Mac. This feels pretty solid at this point.

Current options in the MarcEditor scheduled for the first release

Current options in the MarcEditor scheduled for the first release

MarcEdit Mac MarcEditor Edit Menu wireframe -- options targeted for the first release

MarcEdit Mac MarcEditor Edit Menu wireframe — options targeted for the first release

MarcEdit Mac MarcEditor Reports Menu wireframe showing functions targeted for the first release

MarcEdit Mac MarcEditor Reports Menu wireframe showing functions targeted for the first release

MarcEdit Mac MarcEditor Tools Menu wireframe showing functions targeted for the first release

MarcEdit Mac MarcEditor Tools Menu wireframe showing functions targeted for the first release

May 292015
 

MarcEdit provides lots of different ways for users to edit their data.  However, one use case that comes up often is the ability to perform an action on a field or fields based on the presence of data within another field.  While you can currently do this in MarcEdit by using tools to isolate the specific records to edit, and then working on just those items — more could be done to make this process easier.  So, to that end, I’ve updated the Replace Function to include a new conditional element that will allow MarcEdit to presort using an in-string or regular expression query, prior to evaluating data for replacement.  Here’s how it will work…

When you first open the Replace Window:

Replace Function Changes

Notice that the conditional string text has been replaced.  This was confusing to folks – because maybe that didn’t reflect exactly what was being done.  Rather, this is an option that allows a user to run an instring or Regular Expression search across your entire record before the Find/Replace is run.  The search options grouped below – these *only* affect the Find/Replace textboxes.  They do not affect the options that are enabled when the Perform Find/Replace If…is checked.  Those data fields have their own toggles for instring (has) or regular expression (regex) matching.

 

If you check the box, the following information will be displayed:

New Replace Functionality

Again – the If  [Textbox] [REGEX] is a search that is performed and must evaluate as true in order for the paired find and replace runs.  The use case for this function are things like:

  • I want to modify the field x but only if foobar is found in field y.

 

There are other ways to do this by extracting data from files and creating lots of different files for processing or writing a script – but this will give users a great deal more flexibility when wanting to perform options, but only if specific data is found within a field.

 

A simple example would be below:

Example of the new Replace

This is a non-real world example of how this function works.  A user wants to change the 050 field to an 090 field, but only if the data in the 945$a is equal to an m-z.  That’s what the new option allows.  By checking the Perform Find/Replace If option, I’m allowed to provide a pre-search that will then filter the data sets that I’m going to actually perform the primary Find/Replace pair on.  Make sense?  I hope so.

Finally – I’ve updated the code around the task wizard so that this information can be utilized within tasks.  This enhancement will be in the next available update.

–tr

 Posted by at 11:09 pm

MarcEdit 6 update

 MarcEdit  Comments Off on MarcEdit 6 update
May 252015
 

I’ve been working hard on making a few changes to a couple of the MarcEdit internal components to improve the porting work.  To that end, I’ve posted an update that targets improvements to the Deduping and the Merging tools.

Updates:

  • Update: Dedup tool — improves the handling of qualified data in the 020, 022, and 035.
  • Update: Merge Records Tool — improves the handling of qualified data in the 020, 022, and 035.

Downloads can be picked up using the automated update tool or by going to: http://marcedit.reeset.net/downloads/

–tr

 Posted by at 11:26 pm

2015 OSU Commencement

 Education, Miscellanous  Comments Off on 2015 OSU Commencement
May 102015
 

This weekend, I got to partake in one of my favorite parts of being a faculty member — walking as part of the faculty processional for the 2015 Spring Commencement.  For the past two years, I’ve been lucky enough to participate in the Spring Commencement at The Ohio State University — and while the weather can be a bit on the warm side, the event always leaves me refreshed and excited to see how the Libraries can find new ways to partner with our faculty, students, and new alumni; and this year was no different (plus, any event where you get to wear robes and funny hats is one I want to be a part of).  Under the beating sun and unseasonable humidity, President Drake reminded all in attendance that while OSU is a world-class research and educational institution; our roots are and continue to be strengthened by our commitment as a land grant institution — to be an institution who’s core mission is to educate the sons and daughters of our great state, and beyond.  And I believe that.  I believe in the land grant mission, and of the special role and bond that land grant institutions have to their states and their citizens.  So it was with great joy, that I found myself in Ohio Stadium to celebrate the end of one journey for the ~11,000 OSU graduates, and the beginning of another as these graduates look to make their own way into the future.

In my two years at OSU, one of the things you hear a lot at this institution is a commitment to “Pay it Forward”.  I’ve found among the faculty, the staff, the alumni that number close to 1/2 a million — these aren’t just words but a way of life for so many who are a part of Buckeye Nation.  Is this unique to Ohio State — no, but with so many alumni, the influence is much easier to see.  You see it in the generosity of time, the long-term mentorship, the continued engagement with this institution — when you join Buckeye nation; you are joining one big extended family.

I find that it is sometimes easy to forget the role that you get to play as part of the faculty in helping our students be successful.  It’s easy to get bogged down in the committee work, the tenure requirements, your own research, or the job of being faculty member at a research institution.  It’s easy to take for granted the unique privilege that we have as faculty to serve and pay it forward to the next generation.  Sitting at Ohio Stadium, with so many graduates and their parents and friends…yes, it’s a privilege.  Congratulations class of 2015.

Carol Diedrichs, Vice Provost and Director, The Ohio State University Libraries

I had to catch Carol Diedrichs last Spring Commencement before retirement.

 

Looking into the Stands at the 2015 Commencement

Looking into the Stands at the 2015 Commencement

Looking into the Stands at the 2015 Commencement

Looking into the Stands at the 2015 Commencement…look at all those National Championship banners.

Looking into the Stands at the 2015 Commencement

Looking into the Stands at the 2015 Commencement…there were suppose to be 60-70,000 in attendance. We may love football, but we love our graduates more.