MarcEdit Task Management–planned behavior changes

Below are two messages from a conversation on the MarcEdit list around the Task Manager/Task Functionality in MarcEdit.  I’ve been discussing some changes to the way that this works — particularly related to how the networked task management works.  Since this is a widely used function of the application, I’m pulling these two messages out and making them available here.  If you see something that gives you pause, let me know.

–tr

********************************************************************************************************************************************

Following up — I spent a bit of time working on this and as far as I can tell, everything now works as I’ve described below.  This definitely complicates the management of tasks a bit on my end — but after doing a bit of work — here’s the results.  Please, look over the notes here and below and let me know if you see any potential issues.  If I don’t hear anything, I’ll include this in my update over the weekend.

Simplifying Network Sharing:

When you setup network sharing, the tool now recognizes if your path is a networked path (or something else) and will automatically do a couple things if it’s a networked path:

Adds an import option to the Preferences (option only shows if your path is a network path)

If you click the copy option, it will create a .network option in the local tasks folder and then move a copy of all your items into the network space and into the _tasks.txt file on the network.

On startup, MarcEdit automatically will update your .network task folder with the current data in the defined network folder.

When off-line, MarcEdit automatically will use the data in the .network folder (a local cached copy of your networked data)

When off-line and using a networked path, if you select the task manager, you will see the following:

When you have a networked folder, MarcEdit creates the local .network cache as read-only.  You’ll see this is enforced in the editor.

Changes to the _tasks.txt file — file paths are no longer embedded.  Just the name of the file.  The assumption is that all files will be found in the defined tasks directory.  And the program will determine if there is an old path or just a filename, and will ignore any old path information, extracting just the filename and using it with the defined task paths now managed by the program.

Within the TASKS themselves, MarcEdit no longer stores paths to task lists.  Like the _tasks.txt file, just the name of the task file is stored, with the assumption being that the task list will be in the task defined file.  This means imports and exports can be done through the tool, or just by copying and pasting files into the managed task folder.

Finally — in Windows, networked drives can cause long delays when they are offline.  I’ve created a somewhat novel approach to checking this data — but it means that I’ve set a low timeout (~150 mms) — that means if your network has a lot of latency (i.e., not responsive), MarcEdit could think it’s offline.  I can make this value configurable if necessary, but the timeout here really has an impact on parts of the program because the tasks data is read into the UI of the Editor.  In my testing, the timeouts appear to be appropriate. 

These changes have just been implemented on the windows/linux side right now — I’ll move them to the mac version tonight. 

If you’d like to try these changes, let me know — I won’t release these till the weekend, but would be willing to build a test build for folks interested in testing how this works. 

Finally, I know how much the tasks function is used, so I’m giving time for feedback before going forward.  If you see something described that you think may be problematic, let me know.

Thanks,

–tr

From: Terry Reese
To: MARCEDIT-L@listserv.gmu.edu
Cc:
Subject: Proposed TASK changes…Feedback requested. [Re: Saving tasks lists on the network….]

Following up on this — I’m looking at a behavior change and want to make certain that this won’t cause anyone problems.  Let me explain how the tasks works currently (for access) and how I’ve reworked the code.  Please let me know if anyone is using a workflow where this might be problematic.

Current:

When a task is created or cloned, the task and its full path is saved to the _tasks.txt file (in your local preferences).  Example:

Main task with a whole lot of text is right here               C:\Users\reese.2179\AppData\Roaming\marcedit\macros\tasksfile-2016_03_18_052022564.txt         SHIFT+F1
AAR Library         C:\Users\reese.2179\AppData\Roaming\marcedit\macros\tasksfile-2016_03_18_052317664.txt        
TLA        C:\Users\reese.2179\AppData\Roaming\marcedit\macros\tasksfile-2016_04_19_113756870.txt        

This is a tab delimited list — with the first value being the name, the second value the path to the task, the first being a shortcut if assigned.  The task naming convention has changed a number of times through the years.  Currently, I use GUIDs to prevent any possibility of collision between users.  This is important when saving to the network.

When a user saves to the network, internally, the tool just changes where it looks for the _tasks.txt file.  But data is still saved in that location using the full network path.  So, if users potentially have different drive numbers, or someone uses a UNC path and someone else doesn’t, then you have problems.  Also, when setting up a network folder, if you have existing tasks, you have to import and export them into the folder.  Depending on how deep your task references are, that may require some manual intervention.

If you are off the network and you save to a network path, your tasks are simply unavailable unless you keep a copy in your local store (which most wouldn’t).

Proposed:

Here’s what I propose to do.  This will require changing the import and export processes (trivial), making a change to the preferences (trivial), and updating the run code for the macros (little less trivial). 

First, the _tasks.txt file will have all filepaths removed.  Items will only be referenced by filenames.  Example:

Main task with a whole lot of text is right here    tasksfile-2016_03_18_052022564.txt               SHIFT+F1
AAR Library         tasksfile-2016_03_18_052317664.txt      
TLA        tasksfile-2016_04_19_113756870.txt      

The program already sets the taskfile path so that it can run tasks that are on the network, this will continue that process of normalization.  Now, no file paths will be part of the string, and access to task locations will be completely tied to the information set in the preferences. 

Pros:

• This will make tasks more portable as I no longer have to fix paths.
• Makes it easier to provide automatic import of tasks when first setting up a network location
• Allows me, for the first time, to provide local caching of networked caches so that if a user is configured to use a network location, is offline, their tasks will continue to be available.  At this point, I’m thinking caches would be updated when the user opens MarcEdit, or makes a change to a network task.  The assumption, for simplicities sake, is that if you are using a network drive, you cannot edit local networked tasks — and I’ll likely find a way to enforce that to avoid problems with caching and updates.

Cons:

• If you use a networked task, you won’t be able to edit the offline cache when you are offline.  You’ll still have access to your tasks (which isn’t true today), but you won’t be allowed to change them.
• This isn’t possible with the task manager, but in theory, someone could code tasks to live in locations other than the places MarcEdit manages.  This would go away.  All tasks would need to be managed within the defined network path, or the local tasks folder.
• Using GUIDs for filenames, it shouldn’t happen — but there is always the theoretical chance of duplicate files being created.  I’ll need to ensure I keep checks to make sure that doesn’t happen, which complicates management slightly.

Second — Tasks themselves….

Presently, when a task references a task list, it uses the full path.  Again, the path will be removed to just the filename.  The same pros and cons apply here as above.

For users using tasks, you honestly shouldn’t see a difference.  The first time you’d run the program, MarcEdit would likely modify the tasks that you have (network or otherwise) and things would just go as is.  If you are creating a new networked folder, you’d see a new option in the locations preference that would allow you to automatically copy your current tasks to the network when setting that up.  And, if you are offline and a networked user, you’ll find that you now have access to your networked tasks.  Though, this part represents one more behavior change — presently, if you have networked tasks, you can take yourself offline and create and run tasks local to you.  In the above format, that goes away since MarcEdit will be caching your network tasks locally in a .network location.  If the current functionality is still desired (i.e., folks have workflows where they have are connected to the network for shared tasks, but disconnect to run local only to them tasks), I may be able to setup something so that the task runner checks both the .network and local task directories.  My preference would be to not to, but I understand that the current behavior has been around for years now, and I really would like to minimize the impact of making these changes.

–tr