Printable Version of Topic

Posted by: asavage Mar 16 2013, 05:50 PM

I've noticed this in other versions, but it was really apparent today when I installed v3.71

The last time I used PMV, I was using a networked drive for opening files.

When I installed v3.71 today, the network drive was unavailable. Upon <Ctrl+O>, PMV nearly locks the system. I have three installations, and I thought it was PMV locking up, but it would seem to work right up until I opened the FOC.

The first box opened the FOC, and populated it with placeholder icon, but locked up when drawing the thumbnails. The other two wouldn't even draw the FOC; they just became zombied when <Ctrl+O> and the last drive was a non-available network drive. They would not immediately kill via the Tasklist, but given enough time, they did eventually die.

When I rebooted the server (the network resource for image files), and restarted each PMV, they seem to work OK.

I have noticed bad behaviour in the past WRT the FOC and networked drives, sometimes to the point of having to reboot the client box, and at one time this was bad enough that I just didn't use PMV on networked drives, but the last two years this has been less of a problem, though it would still become unresponsive when <Ctrl-O> on a network drive and I'd have to kill & restart it. In those cases, it would always lock up during populating the icons in the right pane; either it wouldn't draw all the placeholders, or it wouldn't finish creating the thumbnails.

I've put off posting about this bug for years, because it seemed like too much work to track down for what I perceived to be an OS/2 networking-related issue, but I have had it happen at least once on a WinXP SP3 box, just last week. I've no idea about how to approach fixing it, but I thought it would be a good idea to document that at least one user is having a problem WRT the FOC and networked drives.

I do 90% of my work with images on an OS/2 server. Lately, I've been doing a lot more than previously, so the frequency of problems is higher.

[later]
FOC locks up with 3/4 of the FOC saying "can't create thumbnail" and two icons saying "creating thumbnail". CPU at 100%, mouse moves but is showing an OS/2 clock cursor. I was able to grab a camera and take a picture, as it turns out the box had to be rebooted -- the screen remained unchanged for 30 minutes.

[later]
I just updated my WinXP box to v3.71, and saw some odd FOC behaviour too, noted in the second picture below: the left pane does not fully populate all of the mapped drives until about 45 seconds have elapsed.

Here's a couple of pics. If the links don't work, my broadband is down again.
http://asavage.dyndns.org/ftp/hosting/pmview/PMV2000_40.jpg

http://asavage.dyndns.org/ftp/hosting/pmview/PMV2000_41.jpg

I'm off to backlevel to v3.70 (on the OS/2 boxes at least).

Sidebar for the Win version:
While I'm thinking of it: Win version: when I have a whole lot of files on a network drive and the FOC has generated thumbnails, and I dump a new file in that remote directory via another app, then bring up the FOC and refresh (F5), on the Win box there is no indication that the refresh is occurring.

For example, using PhotoShop, I just put those two pics above in a directory I was working on with PMV. I wanted to bring the PSD file in PMV. I opened the FOC and saw all the existing thumbnails. I pressed <F5>. The new PSD file has a filename which sorts it to the end. It took around ten seconds before the new placeholder icon (and eventually the thumbnail) appeared. During that 10 seconds, there is not user feedback that the refresh is occurring. I, as a long-time user of PMV, am used to this, but every so often I forget to "wait for it", and during this interval I'm thinking that something has gone wrong.

As an enhancement, some kind of visual indication that a refresh is in progress would be a good idea.

Posted by: asavage Mar 16 2013, 05:52 PM

Your forum software does not let me edit the Subject, so I cannot fix it.

The Subject (Title) should be "PMV v3.71 blocks . . . ", not "3.17".

Also, the re-capitalization of the Subject words is not nice. That's not what I typed.

Posted by: Peter Mar 26 2013, 10:52 AM

The forum is what it is. The re-capitalization has bugged me for years.

Did you notice the new "Thumbnail Provider" option in v3.71. You may want to set it to "PMView Only" if v3.71 appears slower than previous versions.

The implementation requires a well behaved system. If a drive blocks, then PMView will wait (as you have noticed). I will consider adding the standard windows file open dialog in the next version as a solution to the problem.

Thanks,
Peter

Posted by: asavage Mar 27 2013, 12:04 PM

On the OS/2 side, the wait (in previous releases) was indefinite -- I have literally left it overnight, and still had the FOC locked-up in the morning. I've never had it happen with local drives, only with remote drives on the LAN, so I'd always assumed it was network-related, but as it requires a system reboot to recover use of the system, it's a fairly major inconvenience.

For that reason alone, I'd love to have some kind of non-blocking update of the FOC for the OS/2 side of things. Thanks!

Posted by: asavage Apr 13 2013, 06:39 PM

On my OS/2 box, it's doing it again right now. The FOC is populating the thumbnail icons, the CPU is at 100% and has been for over an hour. The system clock is still running, though it only updates every 4-6 seconds.

The directory that the FOC is loading contains only 50 image files. The directory is on a mapped remote network drive, and is available from other boxes. I can't tell if it's available to the problem box, because the system is unresponsive.

As I mentioned, this has been a problem for several versions. I generally just try to avoid using PMV to work with images on remote drives, but this is not really possible in my work environment anymore.

Is there a debug version that I can use to capture some data that can help you determine why the FOC is running the CPU up to 100% for an hour while processing thumbnails in a directory with only 50 files?

Posted by: Peter Apr 13 2013, 06:45 PM

It sounds like you have overlooked a setting: Have you disabled the OS/2 specific PMView sub-directory scanning option: View->Scan subfolders (right click on the background of the tree)? It is on by default, but may need to be disabled on systems that have slow drives.

That said, exceptionally long processing times as you describe is a common indication that the system is doing heavy memory swapping. Do you have 4G RAM installed? Have you enabled OS/2 High Memory? (Check the System Info dialog in PMView - it will tell you).

Posted by: asavage Apr 13 2013, 07:08 PM

I'll try that in a moment; I had given up on PMV recovering after 90 minutes, so I rebooted, and then tried navigating to the directory again, and PMV again locked up, so I'm rebooting again.

The drives (local & networked) are not particularly slow.

Scanning sub-folders sounds like something I'd want enabled, rather than disabled.

In any event, I've lost work twice today due to this problem.

Posted by: Peter Apr 13 2013, 07:13 PM

It may be a memory issue. Nevertheless, I can't repeat the issue and I definitely do not have the resources to investigate it. Are you on the latest release of eCS?

FWIW, I hear you loud and clear: Time to discontinue the OS/2 version! I know, I KNOW! However, many PMView users encourage me to keep it current with the Windows verision, so I try to do that although I do not have the resources to debug obscure issues under OS/2.

Posted by: Peter Apr 13 2013, 07:24 PM

It comes with a big penalty! This is why the option was added in the first place. Try it, and see if it makes a difference. I added the option because of feedback similar to yours 10 years ago...

Posted by: asavage Apr 13 2013, 07:25 PM

I toggled that option off, was able to successfully save the file in the remote directory, then toggled it back on, and PMV quit.

PMVIEW.LOG:
================================================
PMView Pro v3.71 build 24898 Feb 8 2013
OS/2 20.45 build 14.105 @ 1680x1050, 32 bits
Started Sat Apr 13 17:10:28 2013

IAccessError exception thrown.
function: Unknown
file: ibasectl/ientryfd.cpp
line: 1431.
Error ID is 4097
Error Code group is Presentation System
Exception text is:
EM_SETFIRSTCHAR:UNK 1001 E


IAccessError exception thrown.
function: Unknown
file: ibaseapp/iwindow2.cpp
line: 414.
Error ID is 4097
Error Code group is Presentation System
Exception text is:
EM_SETFIRSTCHAR:UNK 1001 E



Killed by SIGABRT
pid=0x0048 ppid=0x001c tid=0x0001 slot=0x007a pri=0x0206 mc=0x0001
D:\PMVIEW2\PMVIEW.EXE
Process dumping was disabled, use DUMPPROC / PROCDUMP to enable it.
PMView Pro v3.71 build 24898 Feb 8 2013
OS/2 20.45 build 14.105 @ 1680x1050, 32 bits
Started Sat Apr 13 17:22:06 2013

Terminated normally.
================================================

Note: the system date is set correctly:

[D:\UTIL]date
Current date is: Sat 4-13-2013
Enter the new date: (mm-dd-yy)

I do not know why PMV is writing "Feb 8 20113" to the log, but that's what it's doing. I renamed the log and started/stopped PMV, and verified that.

Posted by: Peter Apr 13 2013, 07:27 PM

Feb 8 2013 is the build date of v3.71. Continue reading the log and you will find the actual logging time:

Started Sat Apr 13 17:10:28 2013

The error is an IBM Open Class error. I will see if I can repeat it. Not many users are turning this option ON/OFF. It's a one time setting.

Posted by: Mark_Henigan Apr 15 2013, 09:40 AM

I have had a similar problem. I'm hoping you won't discount it since it is related to a user whose opinion you seem to value. I have extremely large directories under OS/2 and find that I must await PMView's reading of subdirectories before I can again use PMView. But, I need at least a little depth. Is it possible to set it to read a specifici number of directory levels? Too much work? I realize that it may be due to using an underpowered T30 that I'll be upgrading to a W400, or something similar.

Also, I am shocked to hear you talking about stopping development of the OS/2-eCS version. I continue to use it for several hours daily despite the printer problems that you say are not yours to fix, despite my being unable to correct them by reverting to earlier driver revisions. Outside of these fora I only praise PMView. -- Mark

Posted by: Peter Apr 15 2013, 09:51 AM

Read again. I did not say that. However, I have bills to pay as everyone else and a family to take care of. I can't afford to put paid work to the side in order to try to hunt down obscure problems in the OS/2 version.

I will try to keep the OS/2 current with the Windows version, but that's about all I can do. So far I have sold 2 copies of PMView for OS/2 this year.

Posted by: Peter Apr 15 2013, 10:03 AM

Al,

Are you saying that 3.70 works?

That would seem strange, as v3.71 only has a two bugfixes that are not related to the File Open dialog at all.

"New" problems that occur after installing PMView are usually related to OS2.INI corruption (possibly made worse after WarpIN made changes to OS2.INI during installation). Are you regularly cleaning your INI files with IniMaint or a similar tool?

Also, check your setting of the "Use Workplace Shell APIs for copy/move/delete" in PMView's options notebook and read the on-line help for this setting. (This is very important if you don't do regular cleaning of your OS2.INI. Using the WPS API leaves file handles dangling).

Posted by: Peter Apr 15 2013, 10:18 AM

PMView only reads one level down - this is done in order to be able to add the [+] to a directory (to visually show that it can be expanded).

If you turn off subdirectory scanning (which should be much faster), you will lose these [+] signs, but you can still double-click on a directory to expand it.

So if you know your directory structure it's definitely worthwhile to turn off sub directory scanning. However, if you're frequently exploring new unknown paths, the [+] is of course much needed.

Posted by: asavage Apr 16 2013, 12:20 PM

Nothing like displaying my lack of reading skills to the world

I had been planning to do multiple installations and try to find a version that worked "better", but it's taken me quite a bit of time to just get a decently repeatable testcase, so I haven't.

But, to answer your question: the FOC in versions prior to 3.71 could be counted on to go to 100% CPU utilization when reading from a remote drive about one in ten times (a rough estimate). It was often enough that for a long time I just avoided pointing to the FOC to a non-local drive. I've seen the problem for several years.

Lately, I've been doing a lot more work using PMV -- a lot of manual scanning of large materials. My workflow is to use TAME/SANE to scan to a default local directory, open the scan with PMV, rotate & crop, then save to remote drive. That worked fairly reliably, as the FOC would only read the remote directory once -- the first time I'd navigate there to save, and I found that I could avoid the 100% CPU problem that way.

When I went from 3.70 -> 3.71, that didn't seem to work as well: I was having the 100% CPU problem every day. I first noticed a very bad problem the very first time I installed 3.71, because the remote drive was not available, and the ini saves the last opened dir. So, I'd start 3.71, open FOC, and it would try to open a non-available drive, and it locked up there. I do not recall if it was a 100% CPU issue on that particular case. However, it was the same "locked up" with all three installations of 3.71. I backed all three out to 3.70; later, I figured out the trigger, rebooted the remote server, and reinstalled 3.71.

As long as the "last directory read" is available, the FOC does work reasonably well, as well as 3.70, but it does go to 100% CPU more easily than previous versions.

I've found a close-to-repeatable testcase here: if I have a filter of *.* and Thumbnail type of Icon or Mixed, I can usually get it to trigger if I read a certain directory over and over. It takes a lot of reads, but eventually it will trigger and the CPU will go 100%. On both boxes, different versions of OS/2, same local directories and with fewer than (55) files in each.

That makes it not a networking issue (as I'd thought it was for years), not a specific FP level issue, and not a "too many files to read" issue.

On the eCS box, I have TOP/WatchCat (or whatever it's called) installed by default, and I can kill PMV and resume using the box, although if I do that 3+ times other system problems crop up and I have to reboot that box. Since it's JFS, that's not a big deal: it's a fast reboot.

My scanner box is Warp4, does not have WatchCat/TOP installed, uses HPFS386 drives, and a reboot is nearly 15 minutes due to the slow mount of the drives (HPFS & HPFS386 take a very long time to mount large drives), so when the 100% CPU problems occurs, it severely interrupts my workflow.

I've finally gotten a couple of screenshots, one of each box, of the FOC and the CPU monitor.

(click on any image for larger)
http://asavage.dyndns.org/ftp/hosting/PMView/PMV2000_44b.jpg

http://asavage.dyndns.org/ftp/hosting/PMView/PMV2000_45b.jpg

I do have "/log=pmview.log" for both those boxes, but as PMV does not crash --it merely makes the systems unusable -- nothing useful is written to the logs. If there's a way of turning up the log level, I can do that.

I'm typing this on a third box on my network. I've just checked those two boxes whose screenshots I took and processed via CS3, and both are still at 100% CPU and have been for over an hour as I type this. As soon as I submit this post, I'll go reboot them both.

Posted by: Peter Apr 16 2013, 12:41 PM

Aha! I think I know what's going on! When you use a filter of * (or *.*) you make PMView examine all files, and attempt error-recovery on all files if a file format that PMView understands cannot be found.

It is possible that the directory contains a binary file that looks like a valid image format. PMView tries to process it, but the image size PMView reads (from what is believes looks like an image file, but really is not an image file) is so huge that it eats up all your RAM. The machine starts swapping, and voilà - you're at 100% CPU. Everything still "works" fine, but all the machine now does is swapping. Every application will be swapping and control is lost. However, nothing has crashed yet and all is still running "normally". Hence no error in the logs.

Now that you mention it, I have seen these lockups occur with Gigabyte-size ZIP files. My take on this was always "user error" - it was my own fault for leaving these huge files lying around and then use *.* in PMView. In retrospect I realize that I just asked for it to happen! I know what the files are. PMView does not, and goes to work in the hope that it can recover an image and show it to the user!

This DEFINITELY can happen. The solution to this would be to add more restrictions on file detection and maybe the possibility to disable the automatic file recovery feature in PMView. (The conundrum here is the "Bill Gates 640k" issue. What seems unreasonable big today may be normal 10 years from now).

It may also be a bug or unexpected behavior in one of the format detection mechanisms. In this case it needs to be fixed, of course!

In your situation above, I believe it's a problem that can be fixed easily (and needs to be fixed). See if you can pinpoint the offending file. Then copy it to another local directory and see if you can repeat the problem.

Or just copy all files to a different directory first, and once you can repeat the problem delete the files you copied one by one. When the problem goes away, you will know what the offending file is. A good place to start is to first look for files in excess of a certain size (say 50MB) and eliminate the chance that such a file is causing the problem. Yet another way to narrow down the selection is to try using filters like A* and then move on to B*, C* etc until the lockup occurs.

If you find the file that causes the lockup and e-mail it to me, I'll be sure to fix the problem! (Please contact me via e-mail first so I know to expect the file).

Posted by: asavage Apr 16 2013, 03:46 PM

I'd love to be able to hand you a file that triggers this problem, but a single file does not do it. The cause is too subtle for me to determine, given the toolbox I own.

I hate to throw other variables into this, but I've just found another FOC oddity.

I have been able to trigger 100% CPU by using the <CurUp> and <CurDn> in the left pane of the FOC, and changing dirs quickly. I just noticed that if I do this fast enough, after maybe 20-50 directories the FOC "gives up" trying to create thumbnails. It says it's creating thumbnails, but never does.

Thumbnail Type is set to Mixed.

(click on any image for larger)

Correct behavior: "Cannot create thumbnail":
http://asavage.dyndns.org/ftp/hosting/pmview/PMV2000_46b.png

Incorrect behavior: "Creating thumbnail".
http://asavage.dyndns.org/ftp/hosting/pmview/PMV2000_47b.png

Back to the original problem: I copied all the files in that trouble directory to C:\TEMP1 and tried to trigger, but no joy. Then I went back to the trouble directory, and I'm still unable to reproduce. Arrgh. I'll work on it more, but I do not think this is a single-file problem.

These computers have 4Gb RAM installed (max'd out). I know that's a side issue WRT OS/2 and applications. I routinely open and rotate 130Mb PNM files under 1° with PMV on the scan box (11x17" color & 600dpi). It takes a while, but it always works. (Not so, on the identical hardware running XP with 4Mb RAM). When the boxes go to 100% CPU, I cannot operate them directly, but I can still access their HDDs via the LAN, and I intend to inspect the swap file this way. I am thinking that I will not find any swap file growth.

I'll post more when I have more info.

Posted by: Peter Apr 16 2013, 04:03 PM

Al,

Do you have PMView configured to automatically create EA thumbnails? If so, PMView will QUEUE every directory you enter for thumbnail creation. The unexpected result you got above is probably because PMView was still busy working in one of the directories traversed earlier. (In order not to use too much resources, there is only one background thread that creates thumbnails. Files in the current directory are always moved first in the queue, but if the thumbnailer is stuck working on another directory then that does not help).

Posted by: Peter Apr 16 2013, 04:07 PM

Try to use <All Formats> instead of <All Files> and see if that makes the behavior better.

Posted by: asavage Apr 16 2013, 04:08 PM

I should mention that closing and restarting PMV corrects this, until the next time. This is 100% reproducible here.

Hmmm. Those screenshots were on the eCS box. On the W4 box, it says "Unknown file format". That's when PMV is started with /ini=fresh though.

[later]
Ah, fresh.ini = defaults, and Icon Thumbnails is reset to Manual Create. That's the difference.

http://asavage.dyndns.org/ftp/hosting/pmview/PMV2000_48b.png

I don't seem to find a reference to that phrase that explains the difference between it and "Cannot create thumbnail".

Posted by: asavage Apr 16 2013, 04:15 PM

Thumbnail Type = Mixed
Icon Thumbnail = Automatic Create

I think that's Yes.



With "List files of type" set to <All Types> (I cannot see an <All Formats> option) I can still get it to a state where it says, "Creating Thumbnail" and never changes from that.

Posted by: Peter Apr 16 2013, 04:43 PM

Correct. Then my previous post explains the behavior - the thumbnailer and updater are busy with a previously visited folder.




Sorry, I was referring to the 100% lockup in this case. The question is if the behavior is better with <All Types> or if the same kind of problems still appear.

Posted by: asavage Apr 16 2013, 05:58 PM

I've let it sit for several minutes, and the "Creating ..." does not change. I must close PMV and restart it to see any new thumbnails at that point.

However, I can start another instance of PMV and it will load icons fine, for a directory that in the first instance *still* says "Creating...".

These are not particularly slow systems, either.

Posted by: Peter Apr 17 2013, 08:37 AM

Another thing worth trying: Set "Icon Thumbnail -> Manual Create"

Posted by: asavage Apr 17 2013, 11:30 AM

With <All Types>, I was unable to trigger 100% CPU. I tried for about a half-hour yesterday.

Posted by: asavage Apr 18 2013, 10:35 AM

Is there anything else I can do to help narrow-down the area to be repaired?

The FOC is not terribly useful without thumbnails; it becomes a mere file-picker

Posted by: Peter Apr 18 2013, 01:14 PM

Okay. This probably means you have a non-image file that PMView can't handle. The trick would be to find this file and send it to me...



Don't disable the thumbnails completely. Instead use the FOC with on-the-fly thumbnails (i.e. Mixed mode) but disable hard disk writing and generation of hardcopy thumbnails in the EA. In other words, don't enable Automatic Create of EA thumbnails. Manipulation (writing) of EAs is one area where device drivers are commonly buggy and/or not well behaved when it comes to multi-threading.



Yes, the trick would be to find the offending non-image file that causes PMView to think it can read it and causes it to use up all your memory resulting in swapping.

I'm thinking I should probably add a restriction to PMView's file detection that causes PMView to immediately reject a file as valid if the detected image size is bigger than what the installed amount of RAM can handle. Sure, this may cause PMView to say that some valid files are invalid (although they may be perfectly fine, albeit too big to read on the system), but on the other hand this will take care of the problem that non-image files fed to PMView causes it to use up all memory and effectively bring the machine to a halt. On the same note, I should probably add a check that limits error recovery to files smaller than 10MB. The need for error recovery dates back to the 90's when people transferred files over modems, often UU/XX-encoding the files or padding them with a 512-byte header on old Mac systems.

Posted by: asavage Apr 18 2013, 01:59 PM

(click on any image for larger)

"Creating thumbnail".
http://asavage.dyndns.org/ftp/hosting/pmview/PMV2000_47b.png

When it's in this mode, the CPU is not anywhere near 100%. That's what I meant when I'd said that the creating thumbnail process seems to "give up". I'm watching this occur on one of the boxes right now, and the CPU is idling, around 3%, and the FOC shows 49 of 50 thumbnails as "Creating".

If they're all in Creating mode, and the CPU is idle, I don't think it's going to complete. I've been watching it for about ten minutes.

Posted by: Peter Apr 18 2013, 03:08 PM

I just tried to "wander around" the file tree with <All Files> *.* set, and I got PMView to crash 100% repeatably in one directory. I isolated the problem to a .ROM file that PMView thinks is a Kodak Photo CD file (which it is not - it's the firmware for a CNC control).

So I have found a bug that could be the source of the problems you're seeing too. I'll let you know once I locate the source of the problem.

Posted by: Peter Apr 18 2013, 05:03 PM

Ok. I found the problem. It was a Microsoft Visual Studio compiler oddity that apparently appeared with the change from IBM C++ to Microsoft back in v3.10. Now I'm thinking it's not the same problem as you have stumbled on under OS/2. That probably takes a different file to repeat...

I'll see if I can repeat it on OS/2 tomorrow when I have access to an OS/2 system.

Posted by: asavage Apr 18 2013, 08:05 PM

Bugs squashed are always good, even it it's not *my* bug

Posted by: Peter Apr 18 2013, 08:14 PM

Very true... I still believe the core problem is the same: there's a file that lures PMView to believe it can decode it, which leads to disaster. Of course we all want this fixed!

That said, the OS/2 compiler was also switched from IBM to Gnu C++ in v3.10, so at this time it's definitely too early to draw any conclusions. With a little luck, the bug fix might fix an issue in both Windows and OS/2!

Posted by: Peter Apr 19 2013, 09:50 AM

Ok, as it turns out, the bug I found only affects the 64-bit version of PMView. The 64-bit version can't read Kodak PhotoCD (PCD) files at all and crashes when trying to read one. The bug has been there since the first 64-bit release (v3.60). This will be fixed in the next release.

Al, this means that the problem you're having is still unsolved. Clearly it's triggered by a non-image file on your drive. Finding that would be the key to solving the issue.

Posted by: asavage Apr 19 2013, 11:02 AM

I have put hours into trying to find a file or combination of files that will trigger this reliably, and have failed. I've had it happen at least 50 times in the past two weeks, but I cannot present a testcase. I feel that turning up the debug and log level -- what level of detail gets written to the log -- is a method that may reveal clues.

I cannot find a file for you. It's not a single file that causes this, it's some combination of factors -- the number of files in a directory, a number of subdirectories, the mix if image and non-image files.

We have at least two problems:

1) The 100% CPU utilisation, system unresponsive;

2) The thumbnail generation process stopping forever under certain circumstances, with CPU idle. "Creating Thumbnail" forever.

How can I help?

Posted by: Peter Apr 19 2013, 11:24 AM

Al,

We also have determined that the problem only occurs when <All Files> is selected. Choosing <All Types> solves the problem. Is this correct?

Posted by: asavage Apr 19 2013, 12:16 PM

I thought so, but this morning I was proved wrong.

(click on image for larger)
http://asavage.dyndns.org/ftp/hosting/PMView/PMV2000_49b.png

Here's that same directory in normal conditions. It's supposed to look like this:
http://asavage.dyndns.org/ftp/hosting/PMView/PMV2000_50b.png

As I type this, it's been at 100% CPU for about 20 minutes, with <All Types>.

(If you do not see an image above, it's because I am doing server maintenance today, and there will be some outages.)

Posted by: Peter Apr 19 2013, 12:21 PM

Ok. Then the next step is to investigate the possibility that it's an EA issue. Try using the following settings:

Thumbnail Type->On-the-fly
Icon Thumbnails->Manual Create

Thanks,
Peter

Posted by: asavage Apr 19 2013, 06:07 PM

So far, with these settings I cannot get either problem to manifest.

Posted by: Peter Apr 19 2013, 06:16 PM

Excellent! Keep using that setting for a while just to make sure that the problem no longer appears. Then try chaning:

Thumbnail Type->Mixed

I believe this will work fine too. Keep it like this for a few days to make sure everyting works right.

Posted by: asavage Apr 19 2013, 07:16 PM

I have caught up on my work for this batch. Another couple of hours of work should occur next week.

Posted by: asavage Apr 21 2013, 12:04 PM

I had a batch of 128 jpeg files to convert in-place to png yesterday.
The directory contained those files, plus seven .php and two .html files.

Open file name = *_b.*
Type = <Custom>

Thumbnail Type = Mixed
Icon Thumbnails = Manual

During the convert process, the box went to 100%, the swap file remained at 20Mb, and the system ground to a halt. I left it run overnight, but no change by morning. After reboot, I found that it had actually converted 23 of the 128 files.

Posted by: Peter Apr 21 2013, 12:07 PM

Hmm. Now you threw in TWO new factors at the same time, and we don't have a clue which one that caused the problem to appear.

It could be the switch to Mixed, or it could be the usage of the * mask.

Please switch back to On-The-Fly and see if that remains stable.

I'm trying to establish a setting where a problem does NOT occur. Possible problems are:
-Strange file that causes PMView's format auto detection to use up all memory
-Problems with EAs in one or more of the file system drivers on your system

However, a problem with the format detection should normally be repeatable, so it's more likely that this is a problem with EAs.

Posted by: Peter Apr 21 2013, 12:19 PM

Another thing you could try is to disable "Use idle time priority" on the Special page in PMView's options notebook.

Posted by: asavage Apr 30 2013, 12:21 PM

I worked with it several hours yesterday, and with on-the-fly and *.* it did not have the 100% CPU problem (although I got several out-of-memory dialogues, and when it does that it closes).

=============================================
I normally run two PMV windows. I scan two pages at a time on my large-format scanner. I open the scan in one PMV, then copy half of the scan to the clipboard, then in the other PMV I Edit->Paste->As New (<Alt+E,P,N).

Often, the copy to clipboard fails and PMV closes, so I have to open PMV again and load the scan again, this time cropping to 1/2 size and saving a single page to open in the other PMV.

[later edit: actually, if I <Ctrl+X> a small section of the large scan, prepatory to a Paste, that's when I would see this behavior. I think I also saw it on a copy to clipboard, but I'm not so certain about that any more.

In the Win version, on XP SP3, copy to clipboard often silently fails with large copy operations. PMV doesn't say the copy to clipboard fails, but there's nothing to paste elsewhere.]

I have Undo enabled, which doubles the mem requirement. I can't do without Undo.

It would be nice if PMV did not die when it gives the "out of memory" message.
=============================================

FWIW, with 200Mb scan files and on-the-fly for thumbnails, it's a very slow process to work continuously with PMV. The outline of the thumbnails is not shown, nor the file names, until it has read the file (that's apparently what's going on; the actual mechanism isn't apparent), so if I open a dir with, say 1.6Gb of scans, it's minutes until I can even click on one, since there's not even empty thumbnails in the FOC.

I never knew this before, because I never use on-the-fly.

Posted by: Peter Apr 30 2013, 12:52 PM

That's what I'm afraid of: The reason for the hangs you are seeing are most likely because it hangs during processing of EAs (Extended Attributes) in the file system driver. This could explain why I can't repeat the problem here (different system configuration).

Now that you mention it, it's possible that the file system driver works flawlessly under normal conditions but hangs in low memory conditions. Repeating the situation may be difficult.

Actually, when thinking more about this, I guess there's a chance that the problem is in PMView too and is caused by an out-of-memory situation where memory is allocated directly (not via C++ new/delete). I will have a quick look at this and see if some improvements are needed.



PMView was conceived back in 1992 when 32-bit OS/2 was brand new, most machines had 16MB memory or less, and nobody could imagine hitting the 2GB (later 3GB) 32-bit barrier of the operating system. Actually most developers did not realize that this addressing barrier existed, or envisioned it would somehow be solved the day that much memory is needed. (Microsoft fixed it on Windows by moving to 64 bits).

Consequently I designed PMView to assume memory is always available. The "out of memory" message is actually a fairly new addition that was added in PMView v3.24. The message is displayed once PMView catches a C++ runtime out-of-memory exception. At this point it's already too late to do anything fancy.

Today the problem can be solved by switching to 64-bits, so it just does not make sense to spend a lot of time on reworking all existing code to handle the situation. That time is much better spent on adding new features and fixing bugs.



The file names (and thumbnail placeholder outlines, if applicable) should be shown almost immediately UNLESS you have configured PMView to sort files based on a criteria that requires examining the files in detail (file format, image size, image depth, etc.). As long as you sort by a feature that is available by a common "dir" statement, file names and outlines should appear almost immediately. (When EAs are used, PMView store the additional info in the EAs, which is why you can sort on any criteria and still keep it quick).

Thumbnails are added by a background thread, so they should appear one by one (slowly if you have really big files).

Posted by: Peter Apr 30 2013, 01:21 PM

FWIW, this great article may be of some value:

http://www.os2voice.org/VNL/past_issues/VNL0708H/feature_3.html

Posted by: asavage Apr 30 2013, 01:51 PM

You're telling me! It's been quite the chore to try to replicate it here, and when I'm not trying, it always seems to rear its ugly head, esp. when I don't have work saved to disk [grrrrr].

Noted. I can live with it, as on the OS/2 platform it's fairly predictable. That is, it will work for some number of images and operations, and then it will consistently fail, even if I close & reopen every PMV instance, until a reboot. It's still faster than working on the 'doze side, even so.

Even swapping to disk is preferable to losing work, though. If it were possible to catch the situation before the exception handler, a dialogue to "save now"?

Agreed, for this issue.

And I've always used the Mixed setting, so until now I only saw the initial delay upon the first read of the image. But we needed to know if the 100% CPU problem would appear using on-the-fly, so I gritted my teeth and suffered it.

I do not have representative 200Mb images here right now to test, so I can't replicate, and 3.72 is working fine with the stuff I have available, but the lack-of-placeholders was a real problem yesterday. I was pointing the FOC to a local drive and it would take, oh, maybe a full minute to even show placeholders.

To be fair, I do not know if I can blame PMV, as I am running two PMV instances plus SANE scanning in the background, and I/O is very high. CPU is only high when I'm rotating (100% for about 30 seconds to two minutes, depending on the image). I don't expect anything to work snappy when CPU is 100%, but even with high disk I/O I do expect the placeholders to draw.

The problem is that I will be saving an image in one PMV (and that will take about two minutes, because it's saving non-local, and they're large files and one segment between here & there is still 10Mb/s) and while that's saving, I want to load the next image into another PMV instance, but I can't, because the image I want to load was just scanned in, so PMV's never indexed it before. I am frequently 5 to 10 scans behind with PMV -- that is, I can scan quite a bit faster than I can process them in PMV, so let's assume there are five "new" images in the local scan dir and I want to load one in PMV whilst the other PMV is busy saving to the remote drive.

But the FOC will not immediately give me anything to click on. I don't need the thumbnails, the scans are in filename sort order (that's the way TAME & SANE name them), I just need to load the next image. So I wait.

I don't mind the slow thumbnail creation, but I wasn't getting the placeholders fast, either.

I have no new work to scan on the horizon, but next time I do, I'll test this part further.

Meanwhile, do give the 100% CPU problem (or possibly low memory situation) another thought. And, Thanks.

Posted by: Peter Apr 30 2013, 04:02 PM

Thanks to your valuable feedback, and after thinking some more on it, I think I can do a relatively easy fix that will be helpful and serve as an "early warning" before work is lost:

You mentioned the Undo as a possible memory hog, and I think that's a very good hunch! I believe 50% of the time, the undo-memory may be what triggers an out-of-memory condition. I will see if I can make PMView recover from this with only a warning message that lets you continue work. Possibly this will be a good help for preventing lost work...

Posted by: asavage May 3 2013, 11:57 PM

That's a great article. I read it entirely, and while I do not claim to understand it all, it's still enlightening. Thanks for the link.

Posted by: Peter Sep 3 2013, 12:35 PM

Al,

Do you think there's a chance that this problem occurred with the release of v3.68?

v3.68 fixed a 10-year old bug with thread synchronization. This is the only change I can imagine would have an impact on the the FOC behavior. It is possible that the added synchronization brings out new problems if underlying drivers are not well behaved when accessed concurrently by the same application. The added synchronization in v3.68 combined with unexpected resource locking by drivers may possibly result in a deadlock situation.

If/when you have time to try this out, please e-mail me and I will send you links to download v3.67 and v3.68. Comparing these two should give you a clear indication whether the problem is caused by the synchronization added in v3.68 or not.

Another user reported seeing the same problem as you when he upgraded from v3.66 to v3.72. Apparently the problem appeared sometime between these two versions. To me, v3.68 looks like the most likely one, but I need to get this verified before I can work on a solution.

Now, in your original post you said you were going to backlevel to v3.70 which made it sound like this bug is new in v3.71. If this is the case, can you confirm that v3.70 indeed works fine and v3.71 does not? My thinking is that v3.70 has the same problem and that you need to go back all the way to v3.67 before the problem goes away. If I can get this confirmed, then we're half the way to solving the issue...

Thanks,
Peter

Posted by: asavage Feb 14 2014, 09:50 PM

I had an opportunity to test this theory today.

I've been re-creating thumbnail icons as EAs. Several thousand files. I turned on Automatic thumbnail creation and Icon thumbnailing.

I used the FOC to navigate through directories, letting it finish creating thumbnails in each dir before moving on.

With 3.74, it seems reasonably OK unless I open two PMV windows and use two FOCs to do the thumbnail creation. In that situation, one of the FOCs locks up, CPU goes to 100% and the system comes to an effective stop: the clock does not advance seconds.

I have CADH installed, so I can C-A-D, run TOP and find that one instance of PMV is using 100% of the CPU. I can kill it from TOP, and the system resumes normally. The other PMV's FOC continues to create thumbnails fine.

I can start another instance of PMV and can bring the system down again pretty regularly this way, given enough files that need thumbnails created for them.

I installed PMV 3.66 instead of 3.78, and for a while it seemed to not exhibit this behavior. I though you were on to something, but about a half-hour later it did the same thing. Since I was leaving the house for a while, I left it running at 100% for four hours. When I returned, the clock was advancing jerkily (updating every ten seconds or so) but the CPU (Pulse) was still at 100% and no new thumbnails had been created as far as I could tell.

I used CADH & TOP to kill it, and things resumed as normal again.

I can't give you better clues that that.

Posted by: Peter Apr 12 2014, 08:34 AM

Al,

It seems like something in your disk system is deadlocking. Sorry I can't provide more help than that.

Thanks,
Peter