I’ve been spending time this weekend refactoring a major piece of the LibraryFind code partly in an effort to make it easier to add protocol classes. This change affects a lot of the current API code-base, but the biggest change comes in the meta_search.rb file where nearly all the business logic relating to searching, etc. will be removed in favor of a loose plug-in architecture that I’m hoping will make it easier to add additional search classes to the program. However, with all refactoring, there’s a bit of debugging that happens and I tell you, this morning at 4 am, it just wasn’t happening. The big change deals with about 200 lines of code in the meta_search.rb file (which in turn affects the current files that actually make up the protocols and searching). These 200 lines of code have been replaced by the following block:
if is_in_cache == false
_tmparray = Array.new()
objSearch = nil
eval("objSearch = " + _collect.conn_type.capitalize + "SearchClass.new")
_tmparray = objSearch.SearchCollection(_collect, _qtype, _qstring, _start.to_i, _max.to_i, _last_id, _session_id, _action_type, _data, _bool_obj)
if _tmparray != nil: record.concat(_tmparray) end
end
Basically, this code snippet is called if the query isn’t located in the cache. Originally, the code that this snippet replaced was a large case statement that performed different actions depending on what protocol was being utilized. This snippet moves all this logic into models, where search classes are then plugable. The protocol functions will have a naming convention, take the same values (though they’ll do different things with them) and in theory will make it easier to add support for new search types. At least, this is what I’m seeing at this moment as I add the ability to query OpenSearch targets to LF.
Anyway, I worked on this for about 2 hours this morning. The new plug-ins were working fine — items were going into the cache and results were being returned. However, they were being lost during the transition from the API to the UI. Odd. Couldn’t figure out what was going on and debugging is difficult because the application is threaded (another change — a dedicated global thread pump) so sometimes errors occur while other parts of the application are executing. Anyway, at 4 am, I decided to knock off and come back to it in the morning with new eyes to see if I could see what I was doing.
Well, I’m glad I decided to sleep on it. As I was in church this morning and I had an epiphany. Its in lines 7 and 8. The way Ruby’s threading works, variables within the threads are isolated and protected from the rest of the application. To deal with that, ruby has a syntax that allows you to create thread variables that can be accessed outside the application. So for example, if I have a variable that I want to access outside of the thread, I would use something like:
Thread.current[“myrecord”] = Array.new()
This syntax is how plug-ins utilizing the global thread pump will return data to the application. And there was the rub. I’d forgotten that Ruby always returns from a function. Simply for clarity, I always explicitly note what is being returned at the end of each function using the older return syntax:
return xxxxx
I’d conveniently forgotten that feature in the language, and this is what was gumming up the process. The thread pump would finish evaluating the threads, capturing the thread data and then a string or common array would also be returned (outside of the thread pump) which, since not nil, would overwrite the current record variable. Once I had these plug-ins start returning nil values and allow data processing to be handled by the thread pump, all was right in the world again. Unfortunately, I lost 2 hours of sleep last night on this problem, and I’d like to have them back. 🙂
–TR