Pmv 3.17 Blocks On Foc If Last Drive Used Is Unavailable, And sometimes even when it is! Options

[asavage]

I've noticed this in other versions, but it was really apparent today when I installed v3.71

The last time I used PMV, I was using a networked drive for opening files.

When I installed v3.71 today, the network drive was unavailable. Upon <Ctrl+O>, PMV nearly locks the system. I have three installations, and I thought it was PMV locking up, but it would seem to work right up until I opened the FOC.

The first box opened the FOC, and populated it with placeholder icon, but locked up when drawing the thumbnails. The other two wouldn't even draw the FOC; they just became zombied when <Ctrl+O> and the last drive was a non-available network drive. They would not immediately kill via the Tasklist, but given enough time, they did eventually die.

When I rebooted the server (the network resource for image files), and restarted each PMV, they seem to work OK.

I have noticed bad behaviour in the past WRT the FOC and networked drives, sometimes to the point of having to reboot the client box, and at one time this was bad enough that I just didn't use PMV on networked drives, but the last two years this has been less of a problem, though it would still become unresponsive when <Ctrl-O> on a network drive and I'd have to kill & restart it. In those cases, it would always lock up during populating the icons in the right pane; either it wouldn't draw all the placeholders, or it wouldn't finish creating the thumbnails.

I've put off posting about this bug for years, because it seemed like too much work to track down for what I perceived to be an OS/2 networking-related issue, but I have had it happen at least once on a WinXP SP3 box, just last week. I've no idea about how to approach fixing it, but I thought it would be a good idea to document that at least one user is having a problem WRT the FOC and networked drives.

I do 90% of my work with images on an OS/2 server. Lately, I've been doing a lot more than previously, so the frequency of problems is higher.

[later]
FOC locks up with 3/4 of the FOC saying "can't create thumbnail" and two icons saying "creating thumbnail". CPU at 100%, mouse moves but is showing an OS/2 clock cursor. I was able to grab a camera and take a picture, as it turns out the box had to be rebooted -- the screen remained unchanged for 30 minutes.

[later]
I just updated my WinXP box to v3.71, and saw some odd FOC behaviour too, noted in the second picture below: the left pane does not fully populate all of the mapped drives until about 45 seconds have elapsed.

Here's a couple of pics. If the links don't work, my broadband is down again.
Screenshot via camera: OS/2: box locked up at FOC

Screenshot via camera: WinXP: PMV locked for 45 seconds.

I'm off to backlevel to v3.70 (on the OS/2 boxes at least).

Sidebar for the Win version:
While I'm thinking of it: Win version: when I have a whole lot of files on a network drive and the FOC has generated thumbnails, and I dump a new file in that remote directory via another app, then bring up the FOC and refresh (F5), on the Win box there is no indication that the refresh is occurring.

For example, using PhotoShop, I just put those two pics above in a directory I was working on with PMV. I wanted to bring the PSD file in PMV. I opened the FOC and saw all the existing thumbnails. I pressed <F5>. The new PSD file has a filename which sorts it to the end. It took around ten seconds before the new placeholder icon (and eventually the thumbnail) appeared. During that 10 seconds, there is not user feedback that the refresh is occurring. I, as a long-time user of PMV, am used to this, but every so often I forget to "wait for it", and during this interval I'm thinking that something has gone wrong.

As an enhancement, some kind of visual indication that a refresh is in progress would be a good idea.

[Mark_Henigan]

I have had a similar problem. I'm hoping you won't discount it since it is related to a user whose opinion you seem to value. I have extremely large directories under OS/2 and find that I must await PMView's reading of subdirectories before I can again use PMView. But, I need at least a little depth. Is it possible to set it to read a specifici number of directory levels? Too much work? I realize that it may be due to using an underpowered T30 that I'll be upgrading to a W400, or something similar.

Also, I am shocked to hear you talking about stopping development of the OS/2-eCS version. I continue to use it for several hours daily despite the printer problems that you say are not yours to fix, despite my being unable to correct them by reverting to earlier driver revisions. Outside of these fora I only praise PMView. -- Mark

[Peter]

Is it possible to set it to read a specifici number of directory levels?

PMView only reads one level down - this is done in order to be able to add the [+] to a directory (to visually show that it can be expanded).

If you turn off subdirectory scanning (which should be much faster), you will lose these [+] signs, but you can still double-click on a directory to expand it.

So if you know your directory structure it's definitely worthwhile to turn off sub directory scanning. However, if you're frequently exploring new unknown paths, the [+] is of course much needed.

[asavage]

Feb 8 2013 is the build date of v3.71. Continue reading the log and you will find the actual logging time

Nothing like displaying my lack of reading skills to the world

Are you saying that 3.70 works?

I had been planning to do multiple installations and try to find a version that worked "better", but it's taken me quite a bit of time to just get a decently repeatable testcase, so I haven't.

But, to answer your question: the FOC in versions prior to 3.71 could be counted on to go to 100% CPU utilization when reading from a remote drive about one in ten times (a rough estimate). It was often enough that for a long time I just avoided pointing to the FOC to a non-local drive. I've seen the problem for several years.

Lately, I've been doing a lot more work using PMV -- a lot of manual scanning of large materials. My workflow is to use TAME/SANE to scan to a default local directory, open the scan with PMV, rotate & crop, then save to remote drive. That worked fairly reliably, as the FOC would only read the remote directory once -- the first time I'd navigate there to save, and I found that I could avoid the 100% CPU problem that way.

When I went from 3.70 -> 3.71, that didn't seem to work as well: I was having the 100% CPU problem every day. I first noticed a very bad problem the very first time I installed 3.71, because the remote drive was not available, and the ini saves the last opened dir. So, I'd start 3.71, open FOC, and it would try to open a non-available drive, and it locked up there. I do not recall if it was a 100% CPU issue on that particular case. However, it was the same "locked up" with all three installations of 3.71. I backed all three out to 3.70; later, I figured out the trigger, rebooted the remote server, and reinstalled 3.71.

As long as the "last directory read" is available, the FOC does work reasonably well, as well as 3.70, but it does go to 100% CPU more easily than previous versions.

I've found a close-to-repeatable testcase here: if I have a filter of *.* and Thumbnail type of Icon or Mixed, I can usually get it to trigger if I read a certain directory over and over. It takes a lot of reads, but eventually it will trigger and the CPU will go 100%. On both boxes, different versions of OS/2, same local directories and with fewer than (55) files in each.

That makes it not a networking issue (as I'd thought it was for years), not a specific FP level issue, and not a "too many files to read" issue.

On the eCS box, I have TOP/WatchCat (or whatever it's called) installed by default, and I can kill PMV and resume using the box, although if I do that 3+ times other system problems crop up and I have to reboot that box. Since it's JFS, that's not a big deal: it's a fast reboot.

My scanner box is Warp4, does not have WatchCat/TOP installed, uses HPFS386 drives, and a reboot is nearly 15 minutes due to the slow mount of the drives (HPFS & HPFS386 take a very long time to mount large drives), so when the 100% CPU problems occurs, it severely interrupts my workflow.

I've finally gotten a couple of screenshots, one of each box, of the FOC and the CPU monitor.

(click on any image for larger)




I do have "/log=pmview.log" for both those boxes, but as PMV does not crash --it merely makes the systems unusable -- nothing useful is written to the logs. If there's a way of turning up the log level, I can do that.

I'm typing this on a third box on my network. I've just checked those two boxes whose screenshots I took and processed via CS3, and both are still at 100% CPU and have been for over an hour as I type this. As soon as I submit this post, I'll go reboot them both.

[Peter]

I've found a close-to-repeatable testcase here: if I have a filter of *.* and Thumbnail type of Icon or Mixed

Aha! I think I know what's going on! When you use a filter of * (or *.*) you make PMView examine all files, and attempt error-recovery on all files if a file format that PMView understands cannot be found.

It is possible that the directory contains a binary file that looks like a valid image format. PMView tries to process it, but the image size PMView reads (from what is believes looks like an image file, but really is not an image file) is so huge that it eats up all your RAM. The machine starts swapping, and voilà - you're at 100% CPU. Everything still "works" fine, but all the machine now does is swapping. Every application will be swapping and control is lost. However, nothing has crashed yet and all is still running "normally". Hence no error in the logs.

Now that you mention it, I have seen these lockups occur with Gigabyte-size ZIP files. My take on this was always "user error" - it was my own fault for leaving these huge files lying around and then use *.* in PMView. In retrospect I realize that I just asked for it to happen! I know what the files are. PMView does not, and goes to work in the hope that it can recover an image and show it to the user!

This DEFINITELY can happen. The solution to this would be to add more restrictions on file detection and maybe the possibility to disable the automatic file recovery feature in PMView. (The conundrum here is the "Bill Gates 640k" issue. What seems unreasonable big today may be normal 10 years from now).

It may also be a bug or unexpected behavior in one of the format detection mechanisms. In this case it needs to be fixed, of course!

In your situation above, I believe it's a problem that can be fixed easily (and needs to be fixed). See if you can pinpoint the offending file. Then copy it to another local directory and see if you can repeat the problem.

Or just copy all files to a different directory first, and once you can repeat the problem delete the files you copied one by one. When the problem goes away, you will know what the offending file is. A good place to start is to first look for files in excess of a certain size (say 50MB) and eliminate the chance that such a file is causing the problem. Yet another way to narrow down the selection is to try using filters like A* and then move on to B*, C* etc until the lockup occurs.

If you find the file that causes the lockup and e-mail it to me, I'll be sure to fix the problem! (Please contact me via e-mail first so I know to expect the file).

[asavage]

I'd love to be able to hand you a file that triggers this problem, but a single file does not do it. The cause is too subtle for me to determine, given the toolbox I own.

I hate to throw other variables into this, but I've just found another FOC oddity.

I have been able to trigger 100% CPU by using the <CurUp> and <CurDn> in the left pane of the FOC, and changing dirs quickly. I just noticed that if I do this fast enough, after maybe 20-50 directories the FOC "gives up" trying to create thumbnails. It says it's creating thumbnails, but never does.

Thumbnail Type is set to Mixed.

(click on any image for larger)

Correct behavior: "Cannot create thumbnail":


Incorrect behavior: "Creating thumbnail".


Back to the original problem: I copied all the files in that trouble directory to C:\TEMP1 and tried to trigger, but no joy. Then I went back to the trouble directory, and I'm still unable to reproduce. Arrgh. I'll work on it more, but I do not think this is a single-file problem.

These computers have 4Gb RAM installed (max'd out). I know that's a side issue WRT OS/2 and applications. I routinely open and rotate 130Mb PNM files under 1° with PMV on the scan box (11x17" color & 600dpi). It takes a while, but it always works. (Not so, on the identical hardware running XP with 4Mb RAM). When the boxes go to 100% CPU, I cannot operate them directly, but I can still access their HDDs via the LAN, and I intend to inspect the swap file this way. I am thinking that I will not find any swap file growth.

I'll post more when I have more info.

[Peter]

Al,

Do you have PMView configured to automatically create EA thumbnails? If so, PMView will QUEUE every directory you enter for thumbnail creation. The unexpected result you got above is probably because PMView was still busy working in one of the directories traversed earlier. (In order not to use too much resources, there is only one background thread that creates thumbnails. Files in the current directory are always moved first in the queue, but if the thumbnailer is stuck working on another directory then that does not help).

[asavage]

Do you have PMView configured to automatically create EA thumbnails?

Thumbnail Type = Mixed
Icon Thumbnail = Automatic Create

I think that's Yes.

Try to use <All Formats> instead of <All Files> and see if that makes the behavior better.

With "List files of type" set to <All Types> (I cannot see an <All Formats> option) I can still get it to a state where it says, "Creating Thumbnail" and never changes from that.

[Peter]

Thumbnail Type = Mixed
Icon Thumbnail = Automatic Create
I think that's Yes.

Correct. Then my previous post explains the behavior - the thumbnailer and updater are busy with a previously visited folder.

With "List files of type" set to <All Types> (I cannot see an <All Formats> option) I can still get it to a state where it says, "Creating Thumbnail" and never changes from that.

Sorry, I was referring to the 100% lockup in this case. The question is if the behavior is better with <All Types> or if the same kind of problems still appear.

[asavage]

With "List files of type" set to <All Types> (I cannot see an <All Formats> option) I can still get it to a state where it says, "Creating Thumbnail" and never changes from that.

Correct. Then my previous post explains the behavior - the thumbnailer and updater are busy with a previously visited folder

I've let it sit for several minutes, and the "Creating ..." does not change. I must close PMV and restart it to see any new thumbnails at that point.

However, I can start another instance of PMV and it will load icons fine, for a directory that in the first instance *still* says "Creating...".

These are not particularly slow systems, either.

[Peter]

Another thing worth trying: Set "Icon Thumbnail -> Manual Create"

[asavage]

With <All Types>, I was unable to trigger 100% CPU. I tried for about a half-hour yesterday.

[asavage]

Is there anything else I can do to help narrow-down the area to be repaired?

The FOC is not terribly useful without thumbnails; it becomes a mere file-picker

[Peter]

With <All Types>, I was unable to trigger 100% CPU. I tried for about a half-hour yesterday.

Okay. This probably means you have a non-image file that PMView can't handle. The trick would be to find this file and send it to me...

The FOC is not terribly useful without thumbnails; it becomes a mere file-picker

Don't disable the thumbnails completely. Instead use the FOC with on-the-fly thumbnails (i.e. Mixed mode) but disable hard disk writing and generation of hardcopy thumbnails in the EA. In other words, don't enable Automatic Create of EA thumbnails. Manipulation (writing) of EAs is one area where device drivers are commonly buggy and/or not well behaved when it comes to multi-threading.

Is there anything else I can do to help narrow-down the area to be repaired?

Yes, the trick would be to find the offending non-image file that causes PMView to think it can read it and causes it to use up all your memory resulting in swapping.

I'm thinking I should probably add a restriction to PMView's file detection that causes PMView to immediately reject a file as valid if the detected image size is bigger than what the installed amount of RAM can handle. Sure, this may cause PMView to say that some valid files are invalid (although they may be perfectly fine, albeit too big to read on the system), but on the other hand this will take care of the problem that non-image files fed to PMView causes it to use up all memory and effectively bring the machine to a halt. On the same note, I should probably add a check that limits error recovery to files smaller than 10MB. The need for error recovery dates back to the 90's when people transferred files over modems, often UU/XX-encoding the files or padding them with a 512-byte header on old Mac systems.