Ablichter's dashboard

Akilah - well better dangerous than clueless :-) Even when being very late, I feel I have to answer this.
I just tried to explain it a bit more simplified so even coder might understand. From a coder I expect he is able to investigate a problem himself and find a solution or workaround to it. I was able to do so - it should be even easier for you ;-) And yepp, I can be bitchy as well.

Since it seems it matters: I'm an IT professional as well - and my experience is that not only normal user are mixing up processor address space with virtual address space and RAM with virtual memory, etc. pp. ;-) Myself might have helped here, by mixing up terms even when I was talking about PS' VAS all the time.

1) You are right, the memory is virtualized and the whole 2GB user mode is allocated at ones from the beginning - but this says nothing about how it is used by an application.
Having 112MB left as largest free chunk available, when still 615MB mem are free after loading 1GB of images... ...hm, sorry - I won't call that much contiguous ;-)
VMMAP shows this clearly



- mind the white rows and the max free large chunk. The whole user mode looks clustered like this from 0000 0000 until FFFE 0000
If you wonder about the "64" in some of the paths you see in the image:this is a 32bit PS running on a 64bit XP. In order to run it within the same boundaries like on a 32bit system (2GB user mode), I had to remove the largeaddressware flag.

2 & 3) VM, OS' page file
Seems what you expect PS shall do, is not what it actually does. The OS is out of the game here in context of buffering / outsourcing to its page file.
AFAIK PS doesn't let the OS handle its VM on 32bit systems for file access. Instead of using OS disk buffers, its I/Os are directly written to disk / scratch.
This is for a reason - Russel Williams, photoshop architect ones explained why:

1. It costs extra to copy the data to and from the buffers instead of just reading it to / from the disk. If you get data from the buffer instead of doing a disk I / O, this is more than worth it, but...

2. Photoshop’s access patterns to its scratch file mostly don’t match what the OS is expecting. The OS assumes that if you just read or wrote it, you’re likely to need it again soon. But that’s not generally the case with Photoshop’s scratch disk.

So the OS' page file almost stays untouched from PS - actually it is very small, not more than 350MB on my 2GB machine - loading an amount of 2GB images and working on them.
Seems the only amount which is paged to OS' VM (if) are the 160MB for the "image" (see the picture) as you coder IMHO calls the exec, loaded plug-ins and DLLs.

On 64bit OS', with 32bit PS and more than 4GB RAM the memory handling is a bit different from this.

Regards, Ablichter.
Visit our Topas Group at Flickr

Sure - we all paid money for it ;-)
But did you have the latest version of Adjust installed - which AFAIK is 3.2.1?

@Akilah

Ashley, virtual memory can operate well past 2gb

Sure - but on 32bit systems the virtual address space at any time is max 4GB large, regardless how many memory is populated.

2GB out of this 4GB are reserved for the kernel and what is left (minus what is need for hardware, video) have to be shared by all apps.
So there might be not enough RAM left for processing large images, especially when the RAM which is left is not contiguous.

The size of Windows Pagefile (what you call windows virtual memory) is irrelevant, since Windows and PS runs fine without any defined and it isn't used anyway - not even on my 2GB laptop where I can run TA against a 8bit 17500x2500px panorama, but not when its16bit.
Images have to be loaded to and processed in RAM right?

I also believe 4GB pagefile is far to large, usually only ~ 400MB are used / needed. If your computer has a permanent need of a 2GB pagefile, you won't be able to work with it anymore.

Anyway - that's not the point - to solve your prob either reduce image dimensions (resp. merge layers and switch to 8 bit) or go for a 64 bit system.
I know the latter sounds somehow odd, but images will not going to be smaller in future and you'll sooner or later run into this problem also with other apps / plug-ins if you are not willing to work on smaller images.

thanks for looking at that Eric. – Ablichter, on April 25, 2009 20:46

HI Albert, can you please have another look - this time to the mailsystem ? Our friends in the flickr group are complaining they are either not listed in your database or they receive empty (blank) email without keys.

http://www.topazlabs.com/simplify2key... works fine for me - if you get redirected you might clear your browser cache.

try this link http://www.topazlabs.com/simplify2key...

Right virtualguide,

that's how it works. Since PS will use the assigned RAM exclusively soone or later (depending on the number of layers and history states) leaving enough RAM for TA outside PS does the job.
The larger the image the smaller the amount assigned to PS should be, even when it pushes down performance because you hit scratch.

I'm able to process a 2.5GB TIF on a 2GB laptop by this, but I have to lower assigned RAM to ~300MB.

2. On my desktop with AMD AThlon 64 X2 4200+ with 4 GB but XP (32bit = XP understands only something more then 3GB)

Guess you got a 750 MB or 1GB video card?

I'm not much into MACs, but use a simple text-editor.
You have to make sure, the extension (if there is one on MACs) or the header stays the same.

If you read the Tut Harry pointed you to, you will understand how "definitions" within a preset-file works (since the author has no clue at all about MACs, it only was written for PCs only)

And always make a backup before you edit your files ;-)

sorry to hear, if you realize its runs fine on a 2GB laptop, there must be something else with your computer.
Contact me on flickr [at] stock-photogallery.de if you like, but if you do provide some data, like what is the max RAM the system reports (control panel => system) what's the max amount you can assign and what did you have assigned in PS and how many space is left on the harddisk to which you have pointed PSs scratch file to.

and the image size you want to process is needed. In case you load multiple images numbers and total size of them. – Ablichter, on January 17, 2009 10:02

jill,
other plugins might work in a different way. Its like comparing PS with MS Paint.

I hoped I could lead you to a bit better understanding for your system. Because if you understand what happens on your system, how RAM is used and what's its relation to the scratch disks, you would be able to react to different situations adequately by yourself. Adjusting configuration is strongly needed sometimes instead of working with a static system all the time.

The answer to your problem is to find in my reply to ObiJohn (if it tells you nothing: sorry for my poor english): when you understand, that lots of the assigned RAM which once was touched by editing / plug-in "xx", is not freed and what is left might not be enough to run plug-in "yy" next, the consequences is to lower RAM in PS from beginning of a session or to shorten your session length and / or to load smaller images. I prefer the first solution.
The more is not always the better. Even when the opposite was propagated lots of time: give PS only what min. is needed and don't do it the other way round by leaving only the some hundreds MB left for the system.
Maybe it takes 10 seconds for the TP GUI to show up and TP runs >20 seconds longer, but at least you would be able to run TP severall times, instead of shut down PS and start over again, which will take more than 30 seconds.

The other day Albert told us a "housenumber" for that: at least leave 5-6 times the image size RAM untouched by PS. Maybe 4 times the image size would be enough, but better have some buffer.
Or make sure you assign at least 5-6 times the image size you want to process on top of what all loaded images need. As you may see it depends on your workflow. But get prepared TP runs only once then and you have to shut down the session anyway.
No one can tell where the "magick" border for your system (in relation of the image(s) size) is, you have to find out by yourself.

If it helps to make a decision: see the attached image. What you see are two runs of PS (the pink lines), one with 600MB and one with 1.45GB assigned RAM, loading the same 170MB image on a 2GB laptop. Means you should be able to do that as well.

I was able to run TP four times (I could run it 10 times and more, between that USM and lens correction, etc.pp., but I stopped here) and used RAM always dropps down to nice 240MB only.
Whereas with 1.45GB, only the first run finished succesfull, after that RAM dropps to ~580MB.
Any further attempt to run TP allocates even more RAM. Might happpend that from here on also other plug-ins fail.
I think the problem is the difference in committed bytes to available bytes, which is after each launch of TP ~570MB at the first run and only ~130MB after the first launch in the second run.

Another solution for a 32-bit users which won't change to a 64-bit system is the /3GB switch, which would allow applications to assign more than 2GB in PS. But the system engineer in me don't realy like that, because it pushes the OS back to a rather small 1GB corner, cuts down PTEs by 50% and only programs which are coded with the "large_address_aware" flag set, are able to use the larger address space above 2GB.
All other applications don't and while some is not working with PS, the OS / system remains limited to 1GB.
Systems which are not PS-systems only like might get sluggish.

Hope this helps somehow.

@jill, to run 4GB of course does not solve the problem, since the underlaying problem is the 2GB border how Albert explained in his original post:

32-bit XP/Vista only allows each application to use 2GB RAM no matter how much total RAM do you have.

Adobe explains it a bit different - and by this it might make more sense to you:

Photoshop CS2 is a 32-bit application. When it runs on a 32-bit operating system, such as Windows 2000, Windows XP Professional, and Mac OS v10.2.8, it can access the first 2 GB of RAM on the computer. The operating system uses some of this RAM, so the Photoshop Memory Usage preference displays only a maximum of 1.6 or 1.7 GB of total available RAM.

The "first" 2GB (which implies also the term "only") - the rest, or what was left over when the chipset eats its part (see below), is reserved to the OS.
So as you see its a XP / Vista 32-bit limitation - I just wonder that Albert realizes this and still propagates to get more RAM.

While images are growing, 32-bit users are pushed back sooner or later to make use of the scratch disk, no way at all. My largest image is 1.8GB so far ;-)

Latest since 2004 we know:

When the Intel® E7221 chipset is populated to its maximum memory capacity of 4 GB (Giga Bytes), the Operating System (OS) may report a significantly lower amount of available memory. This is only an issue when the maximum memory capacity of 4 GB is used with the Intel Server Board SE7221BK1-E.

Intel

Which is true for almost all 32-bit chipsets, not only the mentioned one. Still.

In other words: lots if not all of the 4th GB evaporates and of course this is not limited to XP - Vista 32-bit faces the same issue.
Dude, Where's My 4 Gigabytes of RAM?

My other system (when booting 32-bit, 4GB) only can use 3GB in total, because of a 1GB graphic card. Wouldn't be a big surprise at all anymore, when I tell you its still the same after I remove the 4th GB?
Albert? Somehow surprised? ;-)

Well, "normal" users which have been told to buy 4GB are of course not aware of this (even when this nowadays is printed in motherboard manuals, at least Asus does this) and I guess it would be hard to understand for them.

So if you want to make use out of your whole 4GB you need to run a 64-bit system - PS 32-bits works fine on it and you can assign a max. of ~3.25GB to it.
By assigning "only" 2.5GB to PS it won't have the negative impact to the OS which it would have when assigning the often propagated 90% or which come with certain "secret tipps" (refering to the /3GB switch).
See another post.

@ObiJohn
when it still comes to errors, lower RAM even more; maybe the rest of your systems consumes some more than others. You have to experiement a bit to find out which is the right value, to get the balance.
To process an image of 170MB in RAM, ~500MB for PS should be enough.

Also be aware that PS doesn't free any (or not enough) RAM.
Consider anything which was assigned also will be used sooner or later while processing and only 33% of the peak will be freed. Sometimes TP runs 2 times without a problem and fails at the third run.
That's the moment when you have to shut down PS in order to free and reorganize RAM.

Any MB which was not initial assigned to PS (and left "outside") can be used by plug-ins, but almost all of it is released after the plug-in finished its work. At least that proves my tests. It may be a bit different but might give you an idea.
So its rather logical for me to assign lesser RAM, when I don't want to get annoyed by hitting the wall twice or more every half an hour, which means to restart PS.
But it also means I have to be the patient man on the world or I need a real fast scratch disk, preferable a RAID 0.

I can run TP on a 370MB PSD on a 2GB notebook as well on my 4GB desktop (RAM makes no differences here at all). I'm able to run it severall times - I get bored after the 5th run...
But on both machines I have to lower RAM to ~ 240 MB - any higher combination, even assigning 100% / 1.7GB, fails.
This makes sense, when knowing TP needs a bit more than 1GB to process the image (measured with perfmon - which btw is much better than the crioppy task manager)
1.7GB minus 240MB assigned, minus 90MB CS4 needs when started, minus ~320MB other processes need, leaves ~1.05GB. voila.
Rising assigned RAM by 100MB: baeng, TP hits the wall on the first run.
For sure this is a large image, so take this just as a numerical example.

The only solution on the longer run for users who want to stick to their 32-bit system and want to process larger files, would be to assign lesser RAM and use more of their scratch space. Of real fast scratch space.

"and have over 1GB free... and that should be more than enough RAM."

No, sometimes its not. On a 32-bit system with 2GB you are able to assign max ~1.5GB (if not less, depends on the hardware) to PS, where 1GB is already very much on the edge. Upgrade to 3 or 4GB will not change much here, because it will rise user mode to max. 1.7GB for PS. You'll win only 200MB.

To load an image takes severall time it own size from assigned RAM.
In case the 168Mb image was not the only one you loaded, load PS only with this image and your present config. Make sure you purge cache when you edited a while and before using TP - if this will not change anything, you have to lower assigned RAM, for the price your system needs to use the scratch disk which will cut down performance.
Lower it to ~800MB and by this you'll leave 700MB to the other programs [b]and[/b] for plug-ins. which should be enough to run a 168MB image.

I can run TP on a 360MB image with several layers by adjusting RAM to the space it really needs to keep the image in RAM, which is around 730MB.

Note that f.e. loading 30 jpgs which sums to a total size of 200MB only, might eat much more of the assigned RAM (>1GB), than a single image which is 200MB big. – Ablichter, on January 13, 2009 22:08

Hi Eric, are you aware that with Simply and Denoise its the same issue? I mean the seperator problem. – Ablichter, on January 01, 2009 15:20

Eric, this happens when already a large amount of RAM was assigned but is still not enough. Had this on 32-bit, 1.5GB assigned and with an image of 540MB with severall layers. DoProc tells it can't fetch the image. Which will of course still come up, when assigning the max of 1.7GB, since the question is: where should the needed mem come from?. I decided to use the /3GB switch than, and I was able to process the image, but this is not a long term solution.

Happy christmas anybody. – Ablichter, on December 24, 2008 20:03

"so problem questions are fine there. "
Like it should be ;-) – Ablichter, on December 23, 2008 22:34

Mary, as far I can see you only got 2GB RAM, which is a bit on the edge with CS4, especially when you load multiple images at the same time.

I know Albert is telling to screw RAM up to 90% which might work on some environments, but often does not, especially when one fills this amount up by loading multiple images and or a huge image.
Try to lower your RAM to 30% or less - before you start CS end all other programs which might run in the background and are not needed and try again.

Extract all files to harddisk instead of calling them from the zip-file and try again. ;-)

Jorge, okay see you there. I left the other Topaz group, since the mod closes down any problem related thread and playing the big censor. Hope this won't happend there.