Feb 282006
 

I just love this…not to diss Western Oregon University, but I visited their home page and the title of the page just struck me as being funny.  It says: “Western Oregon University — Steadily emerging as a leading comprehensive public liberal arts college.” Below, the page notes that they are celebrating 150 years as a university. I almost expected to see next to the 150 year celebration image something saying — celebrating 150 years of mediocrity, but we are getting better, we swear. :)

Here’s the home page (http://www.wou.edu) or a thumbnail of the site:

wou.PNG

Again, not making fun of Western or making a value judgement of the organization — it just struck me as funny.

–Terry

 Posted by at 11:54 pm
Feb 282006
 

Most folks know that MarcEdit 5.0 allows users to localize the application — and in some respects, it does this very differently then most applications. Most applications utilize a resource file to store strings/resources utilized by the application. MarcEdit 5.0 is different — MarcEdit 5.0 utilizes an xml-based language file (or files) and a custom language-engine to help render the language on the screen. Up until now, anyone could edit the language files or create new ones — essentially, they just needed to go to the language folder and edit the xml file that contained your particular language. However, users needed to be comfortable working with the XML file structure. To make this easier, I’ve added a utility that will provide a GUI editor to the language file. Its called the MarcEdit Language File Editor. The tool will allow users to customize language strings and even restore old language files, all from within the comfort of a GUI. Here’s an example of what the tool looks like:

language_editor.PNG

So essentially, it breaks the XML document into a Tree.  Users navigate the nodes, double click on the value and make their changes.  The program will automatically normalize common characters (&, “, <>) — but this was part of the reason why the restore option was included — in case some invalid data is introduced.

I’m hoping some folks will give this a try and let me know what they think.  I have a couple of more ideas on how to enhance this — but I’m definitely going to take suggestions.

Lastly, some of the language files have gotten out of date (in terms of updating them for new dialogs).  I have a small program that I run to sync these files — but it does take a little bit of hands on work — so I’ll try to make time to get these files update.  Also, I’m going back through the application to find the following:

  • Values not in the language files
  • controls that don’t resize correctly in certain cases (since I have a nearly complete russian and swedish language packs, its pretty easy to test most forms)
  • Any invalid calls into the language files (I’ve made some structural changes to the language files — and for the most part, the engine just handled them…I just want to make sure that’s the case).

As always, the program can be downloaded from: MarcEdit50_Setup.exe

–Terry

 Posted by at 11:45 pm

MarcEdit 5.0 update

 MarcEdit  Comments Off
Feb 232006
 

To deal with servers that provide either http headers in invalid formats or servers that use proxies with expired certificates, I ended up having make a few adjustments to the Verify URL utility.  To get an indepth explaination of what changes were made, please see:

Dealing with expired security certificates: http://blog.reeset.net/archives/166
Dealing with invalid http headers: http://blog.reeset.net/archives/167

Update, as always, is found at: MarcEdit50_Setup.exe.

–Terry

 

 Posted by at 10:49 pm
Feb 232006
 

If you have a link checker, you invariably want to just read page headers.  Well, in .NET, they enforce a very strict interpretation of the HTTP header standard.  Unfortunately, a great number of servers (Innovative Interface’s ILS for one), don’t follow the output rules, so .NET’s HTTPWebRequest object will throw a webexception when requesting headers from the server.  Correcting this is fairly easy.  Essentially, you need to build a .config file for your component/application and enter the following configuration data:

< configuration>

< system.net>

< settings>

< httpWebRequest useUnsafeHeaderParsing=”true” />

settings>

system.net>

 

–Terry

 Posted by at 10:43 pm
Feb 232006
 

Some folks that had tried using the Verify URL utility had notified me that it wasn’t reporting a response status when attempting to query materials through an ezproxy url.  The problem it appears, was that the servers in question had either expired security certificates or certificates that were not issued by a trusted site (i.e., hasn’t gone through Microsoft’s verification process).  Well, in .NET, the HTTPWebResponse object cannot accept data by default from a site with an invalid or expired security certificate.  Since this is likely going to be an ongoing problem, I ended up needing to setup a class that would reset the application’s CertificatePolicy object.  To do this, you basically need to setup a class that will accept all Certificates given to it.  Obviously doing this requires a bit of trust — and in this case, since the URLs are coming from MARC records and only being used to read headers–I think we can make that leap.  Anyway, to reset the .NET CertificatePolicy object, you need to setup a class that looks something like this:

internal class AcceptAllCertificatePolicy : System.Net.ICertificatePolicy {public AcceptAllCertificatePolicy() {}

public bool CheckValidationResult(System.Net.ServicePoint sPoint, System.Security.Cryptography.X509Certificates.X509Certificate cert, System.Net.WebRequest wRequest,int certProb){// Always accept

return true; }}

To make use of this function, you need to call this class before you initialize the HTTPWebRequest object with a call like:

System.Net.ServicePointManager.CertificatePolicy = new AcceptAllCertificatePolicy();

–Terry

 Posted by at 10:39 pm
Feb 222006
 

Thank goodness for students.  OSU started their Dspace/Digitization program somewhat in earnest this summer.  And a message from our system admin regarding archival disc usage got me wondering how much scanning actually had been done by the students over the past 6-9 months.  Not counting digitized images (of which, there was ~15,000), the Digital Production Unit has processed 668.2 GB of finished data, with an additional 260 GB currently scanned an in the staging area waiting for metadata. 

Thank god for METS — its been a godsend in terms of having a very simple metadata format to capture the structural metadata of these documents.  While we are using a modified version of the spec — I’m so glad that I decided early to at least encode the archives in something (it was the one part of the plan we really hadn’t thought about too much when we started) — because I’ve had to recreate deriviative files a handful of times from the source — and this has made the process much easier to automate.

–Terry

 Posted by at 10:15 pm
Feb 222006
 

This is one of those tools that I’m going to be working on a bit more (I want to setup a thread pool to speed up the processing), but for now, it works ok. The process allows you to specify within a MARC record what http elements you’d like to check. You check all (and that’s URLs in any record), some or by field. The result is currently a web page that looks like this:

verify_url.PNG

I’m going to be integrating the results with the MarcEditor to allow direct, inline editing of elements when you need to make changes. I just didn’t get that completed tonight.

So where is this function? Look at the main window, under add-ins/verify url. The XSLT that renders the output is found in the XSLT directory, so you can customize it to meet your needs.

–Terry

 Posted by at 2:11 am
Feb 222006
 

MarcBreaker Example:

Dim objMARC

Set objMARC = CreateObject(“MARCEngine5.MARC21″)

‘These files are not provided. You will need to use your own test files. This example is only to show syntax.

lret = objMARC.MarcFile(“C:\Documents and Settings\reeset\Desktop\cyrillic.mrc”, “C:\Documents and Settings\reeset\Desktop\cyrillic.mrk”)

msgbox “here”

The above is the general MarcBreaker function.  This will work with either MARC-8 or the UTF-8 characterset.  If you want to change the characterset, you would utilize the MarcFileEx function which includes a third parameter which specifies the output characterset. 

MarcMaker Example:

Dim objMARC

Set objMARC = CreateObject(“MARCEngine5.MARC21″)

‘These files are not provided. You will need to use your own test files. This example is only to show syntax.

lret = objMARC.MMaker(“C:\Documents and Settings\reeset\Desktop\cyrillic.mrk”, “C:\Documents and Settings\reeset\Desktop\cyrillic.mrc”)

msgbox “here”

Like the function above, this accesses the general MarcMaker function.  This function will work with the MARC-8 or UTF-8 characterset.  If you want to specify a characters, there is a second function, MMakerEx that includes a third parameter that specifies characterset.

Folks might be wondering why I wouldn’t just use two functions with optional parameters (rather than creating a basic and extended version of the functions) — well, its because C# (the language MarcEdit is written in), does not consume parameters marked as Optional (even though the Optional Attribute is valid within the language).  So I’ve worked around it.

–Terry

 Posted by at 2:04 am

MarcEdit 5.0 Update

 MarcEdit  Comments Off
Feb 222006
 

MarcEdit 5.0 beta Update

Ok — I’ve posted a new version of MarcEdit.  This update includes the following major changes:

  • Z39.50 changes — I’ve updated the interface in the single search mode.  When clicking edit settings, the program would continually call he click event (frustrating).
  • Z39.50 changes — I’ve modified the master db a bit and made a couple of updates internally to clean up some code and release resources back to the pc faster.
  • MarcEditor: Updated the Harvester to add the date to and from as items saved on search.
  • Verify URL tool:  Updated the XSLT stylesheet and improved the query overall.  As of this point, the timeout per search is set at 3 seconds for the requesting server to send its http headers…however, I’m thinking I’ll make this value customizable.
  • New help file entries in the For Programmers area.  I’ve also posted an online version of the help to: http://oregonstate.edu/~reeset/marcedit/html/m5/index.html.  This makes use of the XMLHTTPRequest object, demonstrating a light-weight xml/xslt interface.
  • COM components…MarcEdit 5.0 now includes access to the COM components.  This will now enable folks to write perl, python, vbscript, etc. script to MarcEdit 5.0′s MARCEngine utilizing the COM interface.

The program can be downloaded from: MarcEdit50_Setup.exe.  I’m going to be including some vbscripts (as textfiles) for folks to see how the individual functions can be utilized.  In the mean time, I’ll post two example following this post, that show how to use the Making and Breaking function, as well as showing how to use the new Verify Url function.  One more thing, at this point, I’ve only included the MARCEngine in the COM component.  If you’ve looked at the .NET API, you will see that I’ve documentated nearly every ancillary function that’s been made public.  This will eventually be migrated into the COM object.  At this point, I’m just getting the wrappers setup and for the most part, I’ve found the MARCEngine elements are what folks tend to need to use most often.  Finally, I’m working on porting the script wizard at this point and will likely include a wizard that will let you automatically convert scripts written for use with 4.6 to the newer 5.0 syntax.

–Terry

 Posted by at 1:57 am