Jun 302015
 

Changes in this update:

6.1.21
* 6.1.21
** Bug Fix: Conditional Delete - When selecting regular expressions -- there were times when the process wasn't being recognized.
** Enhancement: Conditional Delete - This function use to only work when using the Regular Expression option.  This now works for all options.
** Bug Fix: ValidateISBNs - Process would only process the first subfield.  If the subfield to be processed wasn't the first one, it wouldn't be validated.
** Enhancement: ValidateISSN: Uses mathematical formula to validate ISSNs.
** Bug Fix: Generate Fast Headings (Stand alone tool) -- LDR fields could be deleted.  
** Enhancement: Working to make the global edit functions a little more fault tolerant around record formatting.
** Enhancement: Generate MARC record from URL -- program generates MARC records from Webpages.  If you pass it an LC URL, it will generate data from the MARCXML.

At this point, only the Windows and Linux downloads were updated.  I'll be replacing the Mac download with the first version of the native OSX build the July 4th weekend.  You can get the updates either via the Automated updated tool or from the website at: http://marcedit.reeset.net/downloads

--tr
 Posted by at 8:45 pm
Jun 192015
 

Logistics

Time: 6:00 – 7:30 pm, Friday, June 26, 2015
Place: Marriott Marquis (map)
Room: Pacific H, capacity: 30

Description:

The MarcEdit user community is large and diverse and honestly, I get to meet far too few community members.  This meeting has been put together to give members of the community a chance to come together and talk about the development road map, hear about the work to port MarcEdit to the Mac, and give me an opportunity to hear from the community.  I’ll talk about future work, areas of potential partnership, as well as hearing from you what you’d like to see in the program to make your metadata live’s a little easier.  If this sounds interesting to you — I really hope to see you there.

Acknowledgements:

A *big* thank you to John Chapman and OCLC for allowing this to happen.  As folks might guess, finding space at ALA can be a challenging and expensive endeavor so when I originally broached the idea with OCLC, I had pretty low expectations.  But they truly went above and beyond any reasonable expectation, working with the hotel and ALA so this meeting could take place.  And why they didn’t ask for it — they have my personal thanks and gratitude.  If you can attend the event, or heck, wish you could have but your schedule made it impossible — make sure you let OCLC know that this was appreciated.

 Posted by at 1:24 pm
Jun 172015
 

Coming from the Windows and Linux world — the object where data is copy and pasted from is called the Clipboard.  Not so in OSX.  In OSX, this is referred to as the NSPasteBoard.  Should you need to get string data on and off of it – use the following:

 

private static string[] pboardTypes = new string[] { "NSStringPboardType" };
public void SetClipboardText(string text)
{
	NSPasteboard.GeneralPasteboard.DeclareTypes(pboardTypes, null);
	NSPasteboard.GeneralPasteboard.SetStringForType(text, pboardTypes[0]);
}

public string GetClipboardText()
{
	return NSPasteboard.GeneralPasteboard.GetStringForType(pboardTypes[0]);
}

–tr

 Posted by at 3:23 pm
Jun 062015
 

One of the things that has been a learning experience for me has been understanding how OSX handles threading, understanding what objects are thread-safe, and how to capture user feedback from within an thread running outside of the main UI thread.

Conceptually, Windows, OSX, Linux — UIs tend to be run through a single, primary thread.  This is why long-running processes can lock a window from updating.  In Windows, this can be overcome by creating background threads.  While these threads should not interact with the main UI objects — you do have the ability to spawn a new UI thread that can be used to capture interaction from users.  You just need to be careful to ensure that objects stay compartmentalized since these objects are not explicitly thread-safe.

Within the OSX environment, the UI components cannot be run from outside of the main thread.  This means that if you have a background threading process, you cannot interact with the user because all UI objects are marked as not safe for threading.  Not very practical since there are times when user interaction is definitely necessary.  For example, within MarcEdit’s Mac port — I need to be able to tell users when the program has updated — and I’d like this process to occur outside the primary UI thread so not to slow down application startup or user interactions.  To do this, you need to find a way to thread switch — so that you can encapsulate the code that will interact with the UI around the necessary code to switch execution back to the main UI thread.  In C# and on Windows, you can take non-thread-safe components and using a delegate and Invoke, thread switch within your application.  OSX has something similar, but after hours of looking, I wanted to make sure I document how it works here, so I won’t have to look this up again.  The two functions are: InvokeOnMainThread and BeginInvokeOnMainThread.  These two functions have some limitations — they can only be run within the context of a UI thread (so a window controller because you need access to the NSWindow handle) — but from within this block, you have the ability to generate an NSAlert class (though tricky) as well as generate new NSWindow objects to capture user input.

Here’s an example of how this would work below.  This is the code used for MarcEdit’s Mac notification service.  The program starts a thread when the application loads the main window for the first time — that thread initiates a System.Net.HttpWebRequest object to retrieve information about the current MarcEdit Mac Build.  The program then parses the version information, compares it to the local version and build number — and if the remote build number is newer — generates a UI element that tells the user what has changed, and provides buttons to download the new application.  This code lives in a couple of different classes, but these are the relevant parts.

//MainWindowController

public override void AwakeFromNib ()
{
	this.Window.BackgroundColor = NSColor.White;
	base.Window.Center();
	base.AwakeFromNib ();
				
	System.Threading.Thread objT = new System.Threading.Thread((System.Threading.ThreadStart)delegate { CheckUpdate();});
	objT.Start ();

}

void CheckUpdate ()
{
	helpers objH = new helpers ();
	string data = objH.GetWebContent(clsGlobal.MacNotificationURI);
	string sversion = data.Split (System.Environment.NewLine.ToCharArray ()) [0].Trim();

	if (!string.IsNullOrEmpty (data)) {
		NSObject ver = NSBundle.MainBundle.InfoDictionary ["CFBundleVersion"];
		NSObject ver2 = NSBundle.MainBundle.InfoDictionary ["CFBundleShortVersionString"];

		Version vweb = new Version (sversion);
		Version vlocal = new Version (ver2.ToString() + "." + ver.ToString ());

		if (vweb > vlocal) {
			//To generate a UI element from outside the main UI thread, 
			//use InvokeOnMainThread and create a delegate function block
			InvokeOnMainThread (() => {
				frmUpdateNotificationController objU = new frmUpdateNotificationController();
				objU.Window.Center();
				objU.ShowWindow(this);
				objU.LoadMessage("Build #" + sversion + System.Environment.NewLine + System.Environment.NewLine + 
					data);
			});	
                 } 
	}
}

//helpers class

public string GetWebContent(string uri) {

	System.Net.HttpWebRequest myReq =
				(System.Net.HttpWebRequest)System.Net.WebRequest.Create (new Uri (uri));
	System.Net.HttpWebResponse response = (System.Net.HttpWebResponse)myReq.GetResponse ();
	System.IO.Stream responseStream = response.GetResponseStream ();
	System.IO.StreamReader reader = new System.IO.StreamReader (responseStream, System.Text.Encoding.UTF8);
	string val = reader.ReadToEnd ();
	reader.Close ();
	response.Close ();
	return val;
}

 Posted by at 9:28 am
Jun 062015
 

Having made one preview available for evaluation and feedback, I’ve been diligently working on updating the tool and working on new functionality.  This includes moving onto developing a new notification service so that preview users know when new builds are available and providing a new preferences window to enable support for changing applicable preferences currently exposed within the application.  From the perspective of new work — I’ve begun working on the MarcEditor.  At this point, I’ve mocked out the window and am starting to create the global editing toolsets and connecting actions to the various UI elements.  This is going to be a bit of a time consuming process — but thankfully, it’s been made somewhat easier by the fact that much of the code within the MarcEditor that’s not platform specific, has been moved outside the application to re-usable assemblies.  There’s a bit more refactoring that needs to be done, and I’ll need to re-think how the program streams data into the MarcEditor edit window since the OSX apis make this a bit difficult — but it’s getting there.  Below — you will find screenshots of some of the new work?

–tr

Mac Port Notification Example

MarcEdit Mac Port Notifications Window Example

MarcEdit Mac Preferences: MARCEngine

MarcEdit Mac Preferences window: Current preferences exposed for update are for the MARCEngine and the Automatic Update notification.

This is the initial MarcEditor Wireframe for the Mac.  This feels pretty solid at this point.

This is the initial MarcEditor Wireframe for the Mac. This feels pretty solid at this point.

Current options in the MarcEditor scheduled for the first release

Current options in the MarcEditor scheduled for the first release

MarcEdit Mac MarcEditor Edit Menu wireframe -- options targeted for the first release

MarcEdit Mac MarcEditor Edit Menu wireframe — options targeted for the first release

MarcEdit Mac MarcEditor Reports Menu wireframe showing functions targeted for the first release

MarcEdit Mac MarcEditor Reports Menu wireframe showing functions targeted for the first release

MarcEdit Mac MarcEditor Tools Menu wireframe showing functions targeted for the first release

MarcEdit Mac MarcEditor Tools Menu wireframe showing functions targeted for the first release

May 292015
 

MarcEdit provides lots of different ways for users to edit their data.  However, one use case that comes up often is the ability to perform an action on a field or fields based on the presence of data within another field.  While you can currently do this in MarcEdit by using tools to isolate the specific records to edit, and then working on just those items — more could be done to make this process easier.  So, to that end, I’ve updated the Replace Function to include a new conditional element that will allow MarcEdit to presort using an in-string or regular expression query, prior to evaluating data for replacement.  Here’s how it will work…

When you first open the Replace Window:

Replace Function Changes

Notice that the conditional string text has been replaced.  This was confusing to folks – because maybe that didn’t reflect exactly what was being done.  Rather, this is an option that allows a user to run an instring or Regular Expression search across your entire record before the Find/Replace is run.  The search options grouped below – these *only* affect the Find/Replace textboxes.  They do not affect the options that are enabled when the Perform Find/Replace If…is checked.  Those data fields have their own toggles for instring (has) or regular expression (regex) matching.

 

If you check the box, the following information will be displayed:

New Replace Functionality

Again – the If  [Textbox] [REGEX] is a search that is performed and must evaluate as true in order for the paired find and replace runs.  The use case for this function are things like:

  • I want to modify the field x but only if foobar is found in field y.

 

There are other ways to do this by extracting data from files and creating lots of different files for processing or writing a script – but this will give users a great deal more flexibility when wanting to perform options, but only if specific data is found within a field.

 

A simple example would be below:

Example of the new Replace

This is a non-real world example of how this function works.  A user wants to change the 050 field to an 090 field, but only if the data in the 945$a is equal to an m-z.  That’s what the new option allows.  By checking the Perform Find/Replace If option, I’m allowed to provide a pre-search that will then filter the data sets that I’m going to actually perform the primary Find/Replace pair on.  Make sense?  I hope so.

Finally – I’ve updated the code around the task wizard so that this information can be utilized within tasks.  This enhancement will be in the next available update.

–tr

 Posted by at 11:09 pm

MarcEdit 6 update

 MarcEdit  Comments Off on MarcEdit 6 update
May 252015
 

I’ve been working hard on making a few changes to a couple of the MarcEdit internal components to improve the porting work.  To that end, I’ve posted an update that targets improvements to the Deduping and the Merging tools.

Updates:

  • Update: Dedup tool — improves the handling of qualified data in the 020, 022, and 035.
  • Update: Merge Records Tool — improves the handling of qualified data in the 020, 022, and 035.

Downloads can be picked up using the automated update tool or by going to: http://marcedit.reeset.net/downloads/

–tr

 Posted by at 11:26 pm

2015 OSU Commencement

 Education, Miscellanous  Comments Off on 2015 OSU Commencement
May 102015
 

This weekend, I got to partake in one of my favorite parts of being a faculty member — walking as part of the faculty processional for the 2015 Spring Commencement.  For the past two years, I’ve been lucky enough to participate in the Spring Commencement at The Ohio State University — and while the weather can be a bit on the warm side, the event always leaves me refreshed and excited to see how the Libraries can find new ways to partner with our faculty, students, and new alumni; and this year was no different (plus, any event where you get to wear robes and funny hats is one I want to be a part of).  Under the beating sun and unseasonable humidity, President Drake reminded all in attendance that while OSU is a world-class research and educational institution; our roots are and continue to be strengthened by our commitment as a land grant institution — to be an institution who’s core mission is to educate the sons and daughters of our great state, and beyond.  And I believe that.  I believe in the land grant mission, and of the special role and bond that land grant institutions have to their states and their citizens.  So it was with great joy, that I found myself in Ohio Stadium to celebrate the end of one journey for the ~11,000 OSU graduates, and the beginning of another as these graduates look to make their own way into the future.

In my two years at OSU, one of the things you hear a lot at this institution is a commitment to “Pay it Forward”.  I’ve found among the faculty, the staff, the alumni that number close to 1/2 a million — these aren’t just words but a way of life for so many who are a part of Buckeye Nation.  Is this unique to Ohio State — no, but with so many alumni, the influence is much easier to see.  You see it in the generosity of time, the long-term mentorship, the continued engagement with this institution — when you join Buckeye nation; you are joining one big extended family.

I find that it is sometimes easy to forget the role that you get to play as part of the faculty in helping our students be successful.  It’s easy to get bogged down in the committee work, the tenure requirements, your own research, or the job of being faculty member at a research institution.  It’s easy to take for granted the unique privilege that we have as faculty to serve and pay it forward to the next generation.  Sitting at Ohio Stadium, with so many graduates and their parents and friends…yes, it’s a privilege.  Congratulations class of 2015.

Carol Diedrichs, Vice Provost and Director, The Ohio State University Libraries

I had to catch Carol Diedrichs last Spring Commencement before retirement.

 

Looking into the Stands at the 2015 Commencement

Looking into the Stands at the 2015 Commencement

Looking into the Stands at the 2015 Commencement

Looking into the Stands at the 2015 Commencement…look at all those National Championship banners.

Looking into the Stands at the 2015 Commencement

Looking into the Stands at the 2015 Commencement…there were suppose to be 60-70,000 in attendance. We may love football, but we love our graduates more.

 

 

MarcEdit 6 Update

 MarcEdit  Comments Off on MarcEdit 6 Update
May 062015
 

I’ve posted an update.  The change log is below:

  • Enhancement: Dedup Function: Added support for multiple field evaluation. 
  • Enhancement: Dedup Function: Changed default control number field to 001|019$a
  • Enhancement: Merge Function: Added 019 to the evaluation profile
  • Enhancement: Updated the 001, 010$a, 020$a, 022$a, 035$a process so that it utilizes the same process as the MARC21 process.  This adds multiple field support when handling these individual elements.
  • Optimizations: MARCEngine: character conversion code has been moved into the MARCEngine.
  • Optimizations: Dedup code has been moved into the meedit assembly.
  • Update: JSON View JSON processing assembly has been updated.
  • Update: MARCEngine SAXON dependency has been updated.
  • Behavior Change: MARC Tools UI has been updated to utilize a hybrid of a menu and toolbar.
  • Bug Fix: Windows XP Support has been restored to the installer.

You can pick up the update either using the automated update tool or by going to: http://marcedit.reeset.net/downloads/

–tr

 Posted by at 8:11 pm
Apr 112015
 

This all started with a conversation over twitter (https://twitter.com/_whitni/status/583603374320410626) about a week ago.  A discussion about why the current version of MarcEdit is so fragile when being run on a Mac.  The short answer has been that MarcEdit utilizes a cross platform toolset when building the UI which works well on Linux and Windows systems, but tends to be less refined on Mac systems.  I’ve known this for a while, but to really do it right, I’d need to develop a version of MarcEdit that uses native Mac APIs, which would mean building a new version of MarcEdit for the Mac (at least, the UI components).  And I’ve considered it – mapped out a road map – but what’s constantly stopped me has been a lack of interest from the MarcEdit community and a lack of a Mac system.  On the community-side, I can count on two hands the number of times I’ve had someone request a version of MarcEdit  specifically for a Mac.  And since I’ve been making a Mac App version of MarcEdit available – it’s use has been fairly low (though this could be due to the struggles noted above).  With an active community of over 20,000, I try to put my time where it will make the most impact, and up until a week ago, better support for Mac systems didn’t seem to be high on the list.  The second reason is I don’t own a Mac.  My technology stack is made up of about a dozen Windows and Linux systems embedded around my house because they play surprisingly well together, where as, Apple’s walled garden just doesn’t thrive within my ecosystem.  So, I’ve been waiting and hoping that the cross-platform toolset would get better and that in time, this problem would eventually go away.

I’m giving that background because apparently I’ve been misreading the MarcEdit community.  As I said, this all started with this conversation on twitter (https://twitter.com/_whitni/status/583603374320410626) – and out of that, two co-conspirators, Whitni Watkins and Francis Kayiwa set out to see just how much interest there actually was in having dedicated version of MarcEdit for the Mac.  The two set out to see if they could raise funds to acquire a Mac to do this development and indirectly, demonstrate that there was actually a much larger slice of the community interested in seeing this work done.  And, so, off they went – and I set back and watched.  I made a conscious decision that if this was going to happen, it was going to be because the community wanted it and in that, my voice wasn’t necessary.  And after 8 days, it’s done.  In all, 40 individuals contributed to the campaign, but more importantly to me, I heard directly from around 200+ individuals that were hopeful that this project would proceed. 

Development Roadmap

Now the hard work starts.  MarcEdit is a program that has been under constant development since 1999 – so even just rewriting the UI components of the application will be a significant undertaking.  So, I’m breaking up this work in chunks.  I figure it would take approximately 8-12 months to completely port the UI, which is a long-time.  Too long…so I’m breaking the development into 3 month “sprints”.  the first sprint will target the 80%, the functionality that would make MarcEdit productive when doing MARC editing.  This means porting the functionality for all the resources found in the MARC Tools and much of the functionality found in the MarcEditor components.  My guess is these two components are the most important functional areas for catalogers – so finishing those would allow the tool to be immediately useful for doing production cataloging and editing.  After that – I’ll be able to evaluate the remainder of the program and begin working on functional parity between all versions of the application. 

But I’ll admit, at this point, the road map is somewhat even cloudy to me.  See, I’ve written up the following document (http://1drv.ms/1ake4gO) and shared it with Whitni and asked her to work with other Mac users to refine the list and let me know what falls into that 80%.  So, I’ll be interested to see where their list differs from my own.  In the mean time, I’ll be starting work on the port – creating wireframes and spending time over the next week hitting the books and familiarizing myself with Apple’s API docs and the UI best practices (though, I will be trying to keep the program looking very familiar to the current application – best practices be damned).  Coding on the new UI will start in earnest around May 1 – and by August 1, 2015, I hope to have the first version built specifically for a Mac available.  For those interested in following the development process – I’ll be creating a build page on the MarcEdit website (http://marcedit.reeset.net) and will be posting regular builds as new areas of the application are ported so that folks can try them, and give feedback. 

So, that’s where this stands and this point.  For those interested in providing feedback, feel free to contact me directly at reeset@gmail.com.  And for those of you that reached out or participated in the campaign to make this happen, my sincere thanks. 

–TR