2009 July » Terry's Worklog

MarcEdit testing on ‘Nix (updates)

MarcEdit Comments Off

Jul 312009

So far so good. I’ve had a great response and am starting to get back feedback that I’ll be incorporating into work that I plan on doing this weekend to fix a few ‘Nix issues. But I think that when the next MarcEdit release is posted – I’ll for the first time be including information on how to run MarcEdit on Linux.

Now a Mac – that platform is a completely different animal. Mono is being developed to be cross platform so MarcEdit does run on a Mac – but not well. There are at this point still too many UI issues for me to recommend or suggest someone using a Mac run MarcEdit on that platform. My hope is that Mono 2.6 due in Sept. will correct many of the Mac UI problems. The roadmap indicates that most of the work for this build in the Windows.Forms emulation will be dedicated to Mac UI fixes – so here’s hoping that is true.

–tr

Looking for testers interested in running MarcEdit on ‘Nix

MarcEdit Comments Off

Jul 302009

I’ve started formally looking for guinea pigs that want to give this a whirl. Installation has been simplified, everything works (save the Z39.50 functionality) – so if you would like to try running MarcEdit on either a Linux or Mac (though, running it on a mac may be dodgy – I know that mono development on the mac usually runs a little behind ‘Nix’) let me know. At this time, I’m hoping to get maybe 5 more users (I have 3 now) that are interested in giving this a go (both just installation but also actually using the software).

If you are interested, ping me at terry.reese at oregonstate.edu.

–TR

Last weekend at AALL

Uncategorized Comments Off

Jul 282009

This last weekend I had the opportunity to head out to DC for a couple of days and present on the topic of modifying vendor records using MarcEdit to the American Association of Law Libraries. It was a quick trip. I flew in on Saturday and was out by Monday, but one that I enjoyed for a number of reasons.

First, what did I do? Well, I gave two talks, if you will. The first was on using MarcEdit to deal with vendor records. Given the length of time allotted (60 minutes) – I focused primarily on two main aspects of the program – the MarcEditor and the Delimited Text Translator. The goal of the session really was to help attendees be able to leave the room with the knowledge that they would need to start editing records right away. Hopefully, that was accomplished. However, this talk also represented a first for me, in that this was the first time that I gave a presentation on MarcEdit while running MarcEdit natively on Linux. This is something that I do at work because I can live with a few of the less polished aspects of the program – but over the past month or so, I’ve been doing work on MarcEdit to specifically address those areas in need of polish on non-Windows systems and this weekend was kindof the coming out party. I know that this will make a number of people happy (a small vocal few that have been asking how to run MarcEdit on non-windows systems). I know the first question I may get in regards to this development is, how can I run MarcEdit on my [insert non-windows] system? And to those with that question, I’m going to ask for more patience. There are 4 or 5 significant areas where I still want to correct before I officially start hosting and posting instructions on how to run MarcEdit on a non-windows platform. Some of these things are simply UI things to update, others are related to ensuring that MarcEdit’s many ancillary tools can be initiated. But I am working on it, and give my “test�? run this weekend, I’m a lot more comfortable with how the program looks and runs than ever before.

Anyway, the second talk that I give was really more of a question and answer period. I hosted a round-table on MarcEdit, allowing people with specific questions or issues to come and ask their questions. This was also a good time for me, because I get a chance to talk with folks using the program and get a feel for the types of problems that they may or may not be having with the application – as well as get some great ideas for where I might make improvements.

Oh, and while I was at AALL, I did take a chance to drop into a few of the sessions though I quickly realized that I was out of my element when the discussion turned to the bar, laws and codes in general.

All and all, it was a nice trip. I learned a few things, got some great feedback, and had an opportunity to spend a little time in our Nation’s capital (a place that never fails to leave me inspired).

–TR

Supporting Right to Left languages in MarcEdit (part 3)

MarcEdit Comments Off

Jul 262009

I’ve been spending a bit of time over the past few weeks doing some travelling but also working on this problem in MarcEdit. When I left it last, I was pretty close to having a solution – the only thing that had been still a little wonky was dealing with data elements that were primarily mixed numeric/text delimiter pairs. Well, the good news is that it looks like I’ve got it figured out. As noted earlier, the bidirectional algorithm provides methods for inserting control codes that can be used to distinguish embedded data to ensure that the data is handled correctly (or, handled in the fashion that the user is expecting). I think I’ve gotten to that point with MarcEdit.

Now, in order to get this to work, I had to implement a couple of changes. I’ve added spacing between delimiters and the field data that make them up. These spaces are cosmetic and are filtered out when data is saved. It is only for the purposes of rendering the data correctly. Likewise, the LDR statement in MarcEdit, for the purposes of the Right to left translation, when streamed to the screen, the field is swapped from LDR to 000. In MarcEdit, field 000 has always been a synonym for the LDR field – however, for the purposes of correct right to left rendering, changing this to a numeric representation made life much easier.

This weekend, I’m wrapping up a build of this for folks interested in testing. I have to dedicated testers (both in the middle east) who will be working with the software and providing feedback – specifically as it relates to rendering and use of the global utilities. Because this is nearly finished and there haven’t been any outstanding [blocking] issues, I’ll be holding the next update till this work has completed.

I’ll put a small write-up in the tutorial when this ships, but how will this work in MarcEdit? It will actually be very simple to activate.

1) First, you start with a record and open it in the MarcEditor:

2) Using either the keycode combination of: CTRL+SHIFT+R or right clicking on the menu and selecting Display Right-to-Left from the menu:

3) When you select that option, MarcEdit will re-load your data file, shifting the display to Right to left.

My guess is that as my testers get their hands on this, we will run into a few issues (there always are) – but so far, it appears from my untrained eye (using the sample data provided to me over the past two weeks) that everything appears to working as one might expect.

–TR

Boys’ last track meet of the season

Family Comments Off

Jul 262009

Wed., the boys had their last track meet of the season. Well, officially, Kenny had his last track meet of the season. Nathan isn’t technically in track, but they have an open 4 and under division that we let him pick 3 events to participate in. Of course, that means that Alyce and I get as much exercise as the boys since we have to shuttle them from event to event.

Nathan decided to do the same events as Kenny, so a lot of times, they got to watch each other do their specific events, which were the 50m, the long jump and the Frisbee throw (I guess, that would be like the discus in real track). I have to admit, I enjoy watching the boys participate in track because I remember what it was like to do many of these events myself. When I was in high school, I would have participated in the decathlon if they would have had it – but they didn’t in our area – so I often did a wide variety of events. But the one’s that I had the most success in were the sprints (100 and 200) and the jumps (Long Jump and Triple Jump). The triple jump was always my favorite, but I was a fair long jumper (over 20 ft) – so I’ve been having a good time working with Kenny this year trying to teach him some simple techniques to help him move from simple stepping into the pit to actually jumping into the pit.

So how did they do? Well, they came home with a cornucopia of ribbons. Kenny finished first in the Frisbee throw, 3rd in the 50 m and 6th in the long jump. However, in the long jump, Kenny really made a lot of progress. He PR’d during the meet (7’ 8�?) and seems to be getting a much better grasp of how you jump into the pit. Over the last two meets, he’s made a lot of progress – so I’m excited to see how he does next year.

Nathan on the other hand, finished 6th (which he as really happy about because it was a yellow ribbon) in the Frisbee throw, 2nd in the 50 m and a participants ribbon in the long jump.

The best part was that they both had a really good time at the meet and unlike the weather predictions, the temperature stayed in the low 80’s (rather than near 100 like advertised) so we all didn’t swelter in the heat.

The meet also gave me a chance to try out the YouTube integration on my Google Phone. I uploaded short snippets of each of the boys doing each of their events on my phone and was able to upload them directly to YouTube. They turned out ok…I’ve posted them below:

Kenny’s Events:

Frisbee Throw

Long Jump

50 m

Nathan’s Events:

Frisbee Throw

Long Jump

50 m

–TR

Right to left rendering Part II

MarcEdit Comments Off

Jul 092009

So, a couple of a additional notes on this. I got some feedback from my latest attempt and while I’m closer, there are still some significant issues. Also, my testers and I weren’t seeing the same things on the screen when testing so I thought I’d follow up on two aspects of working with Right to Left languages that are not completely intuitive.

1) While Windows does support Right to Left rendering by default, it will not apply the bidi algorithm correctly unless you have explicitly turned on support for complex scripts (at least in XP-, having check my vista or Windows 7 testing boxes yet). This wasn’t obvious. It seemed to me that if I could see the text, could switch on the Right to Left processing, that certainly, I would be seeing what an Arabic user would be seeing. Well, no. If you want to actually have Windows support correct rendering of Right to left data, you need to open the Regional and Language Control Panel Applet and check the Install files for complete script and right to left languages:

Obvious – right? Once I dug up my Windows XP disk, I was able to install the support files and now, I can see what the same thing that an Arabic user would see. So, at least we are now working with the same screwed up display.

2) The bidi is an interesting algorithm, and the more I’ve been looking at it, the more that I think that options actually exist in the algorithm to solve my problem. At issue is how the algorithm treats data that shouldn’t be treated as right to left or mixed displays. The best explanation that I’ve read comes from here: http://blogs.msdn.com/vsarabic/archive/2008/03/24/mixed-time-date-display.aspx#comments explaining why mixed Time/Date elements often display incorrectly within Arabic interfaces. It explains how numbers and neutral characters are processed and how the rendering shifts when non-neutral, non-arabic characters are encountered. For MarcEdit’s purposes, I have a pretty good idea which non-neutral characters are causing my problems – the delimiters (since these can be a-z0-9). But the bidi (which you can read more about here: http://unicode.org/reports/tr9/) includes a set of character overrides that can be embedded to force certain data to be interpreted as strong, weak or neutral characters for processing. Playing around with this a little bit, I found that I could embed a few key elements and change the output to look like the following:

Still not perfect, but getting much closer to what I’m looking for. Of course, embedding these character codes invalidates the MARC record so they would need to be filtered out when saved or the saved data would be a mess – but I think that this could potentially be useful, specifically for people doing web display – since these embedded characters would allow you to essentially mark control/display data so that the rendering doesn’t affect the overall output of the text. Make sense? Not much to me either.

–TR

Supporting Right to Left languages in MarcEdit

MarcEdit 5 Responses »

Jul 082009

Life is a funny thing. I’ve never been someone with a distinct desire to understand the intricacies of language or characters. That’s stuff that my wife enjoys – she’s the linguist while I’ve barely mastered English.

Yet, here I am, thanks to MarcEdit, having to become much more familiar with character sets and language than I’d ever dreamed. Some of these issues are easy. Character sets for example. There are actual solutions to the character set issues (both moving and identifying). In MarcEdit, I found a long time ago that I could appease the math geek inside of me by forgoing the creation of large translation tables and coming up with mathematical translations for moving data between various character sets. It’s not perfect (i.e., there are exceptions that require the use of a lookup table) – but it’s slick enough that in MarcEdit all language translations (save CJK) are handled through a relatively simple formula with support for exceptions.

Language on the other hand stymies me. Lately I’ve been having conversations with Arabic and Hebrew users that would like to be able to support Right to Left processing of their language in MarcEdit. In general, support for this is built right into the operating system through a localized implementation of the Unicode Bidirectional (BiDi) Algorithm. So, in MarcEdit, you can mark the MarcEditor window and have it shift to the OS supported output. The problem comes from MARC and the notepad like format MarcEdit uses for editing. Because of how numbers, punctuation and mixing Latinate and non-Latinate characters are handled in the algorithm, the output presented to the user is close – but not perfect. For example, in Arabic:

This is a window from a colleague in Dubai that has been helping me wrap my head around the changes needed to make MarcEdit easier to use for the creation of non-English records.

The problem that we run into specifically has to do with numbers and words. Many of the switched elements are elements that have numbers (or groups of numbers) attached to them – which is problematic because there is a lot of numerical data in MARC.

I’ve been pounding away at this, and I’m hoping that I might have come up with a workable solution. It’s required writing a custom implementation of the Unicode Bidirectional (BiDi) Algorithm – though, I’ll admit, at this point, it’s actually a very simplified version so I’m going to need to send this to a few people to make sure that it doesn’t butcher some general assumptions. But with a little bit of work, I’ve been able to get an output that looks something like this:

Of course, since I’ve change the output algorithm, I’ve had to create a method for reassembling the data so that it gets placed back into an order that MarcEdit can compile into MARC. I’m certain that there is still more work to be done, but the upside is that very soon, MarcEdit should be able to fully (or at least, more closely) provide better language support for any user utilizing a Right to Left rendering language. Will I understand the whole language issue any better, probably not – but I guess that’s as it should be.

–TR

A long weekend reprised

Family Comments Off

Jul 062009

I hope everyone had a nice 4th of July weekend. We had a good one. Our little town of Independence, OR goes all out on the 4th. They have two fireworks celebrations (one for the locals, one for everyone else) – they have a mini-marathon, a large parade and a carnival that goes on for 4 days. So we had ample opportunities to get outside and enjoy the sunshine (90+ degree days), the fair food and the fireworks.

The last day of the carnival, my wife and I took our boys down to the carnival for a day of unlimited rides. They were kiddie rides, and Kenny and Nathan loved different types of rides. Kenny loved the long slide and Nathan loved the kids roller coaster. Here’s a few pictures from our day.

–TR