low light

On Mar 25, 6:52 am, Paul Furman wrote:


I finally took a shot where I wished I'd turned off RAW compression on
my D200. It was the new moon, shot mid-day almost straight up, kinda
hazy at +2 EC just before blowing then darkened in PP to a black sky and
the remaining moon detail was pretty badly posterized. I actually got it
to look good with a lot of PP work so I can't easily show the problem
but I guess that was the cause. A rather unusual situation.

That's interesting; I never managed to see any difference between
compressed and uncompressed raw. Even when I tried to force it (by
unrealistically extreme processing) I couldn't see it, even by
subtracting the images in photoshop. Is it easy for you to post this
somewhere? From what you say, it sounds like you did some heavy
processing, did you do it in 16 bits or 8 (I mean after conversion)?
This sort of extreme adjustment is just about the only place where I
can see a difference between 8 and 16 bit processing (or 15 bit or
whatever it is that photoshop actually uses).

On the one hand, I find it hard to believe it's the compression, the
gaps between the levels that are present are smaller than the
theoretical photon noise, so basically the extra tonal resolution of
uncompressed raw just records noise more accurately [and since you
can't really see shot noise in reasonably high-key areas, that tells
you it's irrelevant resolution anyway]. On the other hand, who knows?
Maybe there is some indirect effect.

John Sheehy wrote:
Almost every reply you or Roger has made to me has ignored what I have
actually written, and assumed something else entirely.

Look at the post you just replied to; I made it quite clear in my post
that Roger responded to, that the effect only happens with *ONE CAMERA*,
yet Roger replied as if my technique were at fault, in some elementary
way. He didn't pay attention, and *YOU* didn't pay attention, made
obvious by your ganging up with him and failing to point out to him that
it only happened with one camera.

Did you even notice that fact? (That post wasn't the first time I said
it was only one camera, either).

I look at the big picture. It's not just one line of one of
your responses that I have been responding to.
Here are some of your posts, which involve MULTIPLE cameras:

You said:
The results vary from camera to camera as well; my 20D and my FZ50 have no
such limit to S/N, but the XTi does.
and responding to data I've presented:
Those 10D figures are way off. They are 1.9, 2.8, 4.9, 9.0, and 18.0.
Perhaps your figures were taken from a blackpointed RAW blackframe.
and:
I don't recall seeing values this low at the low ISOs in the Nikon RAW
files I had.
and data others have derived using the same methods I use:
The 5D figure is very high for ISO 1600, also. The 5D ISO 1600
blackframes I have here are all 4.6.
and then you discuss conclusions from other cameras:
Here's the shadow area of a 1DmkIII ISO 100 RAW, at the original 14 bits,
and at quantizations to 12, 11, and 10 bits:
http://www.pbase.com/jps_photo/image/76001165
The demoasicing is a bit rough; it's my own quick'n'dirty one,

and these are just from a coulple of your many posts in this thread.

What I see is you attacking the data on multiple cameras from multiple
sources, all of which paint a consistent picture. But as the details
of your own testing come out, and shown to be inadequate,
you start the personal attacks. A more appropriate response
would be to 1) verify that your methods actually do not suffer
from the problems I outlined, and 2) then explain why your results
with your methods are actually correct and why they are better
than those using industry standards.

Roger

John Sheehy wrote:
"Roger N. Clark (change username to rnclark)" wrote
in :


Well, lets look at this another way. Go to:
http://www.clarkvision.com/imagedetail/dynamicrange2

4 bits is DN = 16 in the 0 to 4092 range. In 16-bit
data file, that would be 16*16 = 256.

Now go to Figure 7 and draw a vertical line at 256 on the
horizontal axis. Now note all the data below that line that
you cut off. Now go to Figure 8b and draw a vertical line
at 4 stops, and note all the data you cut off. Now go to
Figure 9D and draw the vertical line at 256 and
note all the data you cut off. (Note too how noisy the
8-bit jpeg data are.)

You can't just divide by 16, to drop 4 LSBs. 0 through 15 become 0. You
have to add 8 first, and then divide by 16 (integer division), then
multiply by 16, and subtract the 8, to get something similar to what you
would get if the ADC were actually doing the quantization.

Fair enough, I'll redo the test.

Here is the full set of images:

See figure 9 at:
http://www.clarkvision.com/photoinfo...ht.photography

Here is the original raw data converted linearly in IP, scaled by 128:
http://www.clarkvision.com/photoinfo...s128-876px.jpg

Now here is the same data with the bottom 4 bits truncated:
http://www.clarkvision.com/photoinfo...s128-876px.jpg

Now here is the same data with the bottom 4 bits truncated, doing nearest integer
using your formula. While subjectively it looks a little better, it has still
lost a lot of image detail compared to the full 12-bits:
http://www.clarkvision.com/photoinfo...s128-876px.jpg

You lose quite a bit in my opinion.
It would be a disaster in astrophotography.

Roger

The ADC is
working with analog noise that dithers the results; you lose that
benefit" when you quantize data that is already quantized. You won't
notice the offset when the full range of DNs is high, but for one where a
small range of DN is used for full scene DR, it is essential. I am
amazed that you didn't stop and say to yourself, "I must have done
something wrong" when you saw your quantized image go dark. That's what
I said to myself, the first time I did it. I looked at the histograms,
and saw the shift, and realized that an offset is needed unless the
offset is a very small number relative to the full range of the scene.

In the case of the mkIII image at 14, 12, 11, and 10 bits in another
post, I used PS' Levels, because it simplifies the process, by doing the
necessary offset to keep the distribution of tones constant.

On 25/3/07 17:00, in article ,
"Lionel" wrote:

On Sat, 24 Mar 2007 14:00:08 GMT, John Sheehy wrote:

"Roger N. Clark (change username to rnclark)" wrote
in :


Well, lets look at this another way. Go to:
http://www.clarkvision.com/imagedetail/dynamicrange2

4 bits is DN = 16 in the 0 to 4092 range. In 16-bit
data file, that would be 16*16 = 256.

Now go to Figure 7 and draw a vertical line at 256 on the
horizontal axis. Now note all the data below that line that
you cut off. Now go to Figure 8b and draw a vertical line
at 4 stops, and note all the data you cut off. Now go to
Figure 9D and draw the vertical line at 256 and
note all the data you cut off. (Note too how noisy the
8-bit jpeg data are.)

You can't just divide by 16, to drop 4 LSBs.

Of course you can.

0 through 15 become 0. You
have to add 8 first, and then divide by 16 (integer division), then
multiply by 16, and subtract the 8, to get something similar to what you
would get if the ADC were actually doing the quantization.

What a complete load of crap. Have you *ever* worked with ADC's in
your life?
It sounds like you might be confusing 2 quadrant ADC's that're used
for audio applications with single quadrant ADC's that're used for
this sort of device.

'Dropping LSB's'? 'Quadrant ADC's'? My brain's fallen out.

Are there any real photographers here?

acl wrote:

On Mar 25, 6:52 am, Paul Furman wrote:


I finally took a shot where I wished I'd turned off RAW compression on
my D200. It was the new moon, shot mid-day almost straight up, kinda
hazy at +2 EC just before blowing then darkened in PP to a black sky and
the remaining moon detail was pretty badly posterized. I actually got it
to look good with a lot of PP work so I can't easily show the problem
but I guess that was the cause. A rather unusual situation.


That's interesting; I never managed to see any difference between
compressed and uncompressed raw. Even when I tried to force it (by
unrealistically extreme processing) I couldn't see it, even by
subtracting the images in photoshop. Is it easy for you to post this
somewhere?

Here's a 'bad' curves version, what I got out of the raw converter & the
original:
http://www.edgehill.net/1/?SC=go.php...007-03-22/tech
-the final is up one folder
I'll email the NEF file if you want to tinker, just remove the hyphens
from my email. In the end I did salvage it pretty well just using ACR &
8 bit photoshop.

From what you say, it sounds like you did some heavy
processing, did you do it in 16 bits or 8 (I mean after conversion)?
This sort of extreme adjustment is just about the only place where I
can see a difference between 8 and 16 bit processing (or 15 bit or
whatever it is that photoshop actually uses).

On the one hand, I find it hard to believe it's the compression, the
gaps between the levels that are present are smaller than the
theoretical photon noise, so basically the extra tonal resolution of
uncompressed raw just records noise more accurately [and since you
can't really see shot noise in reasonably high-key areas, that tells
you it's irrelevant resolution anyway]. On the other hand, who knows?
Maybe there is some indirect effect.

teflon wrote:
Are there any real photographers here?

How's this?
http://www.clarkvision.com/galleries

teflon wrote:
[]
'Dropping LSB's'? 'Quadrant ADC's'? My brain's fallen out.

Are there any real photographers here?

Obviously there are, and ones who wish to have a better understanding of
the equipment used. If you are uncertain about terms, why not ask or look
them up?

David

"acl" wrote in
oups.com:

On the one hand, I find it hard to believe it's the compression, the
gaps between the levels that are present are smaller than the
theoretical photon noise,

That still posterizes the noise and signal a little bit. You're not
likely to see it with any normal tonal curve; you really need to increase
the contrast quite a bit, and you will see it. For example, I remember
shooting in extreme fog a couple of years ago, where I used +2 EC with my
20D, at ISO 400, and raised the effective blackpoint such that the dark
parts of the Robins approached black. It brought up a bit of noise that
would not normally be seen, with any exposure compensation level, while
black was still anchored at black. Same with taking pictures of things
reflected in glass over a white background, if you try to restore black
in the processing.

so basically the extra tonal resolution of
uncompressed raw just records noise more accurately [and since you
can't really see shot noise in reasonably high-key areas, that tells
you it's irrelevant resolution anyway]. On the other hand, who knows?
Maybe there is some indirect effect.

Recording noise better is a good thing, and the same conditions record
signal better as well (and allows the brain and algorithms to separate
them better, as well).

In this particular case, it is only likely to be seen in extreme
blackpointing, or perhaps extreme sharpening.

--


John P Sheehy

On Mar 27, 2:39 am, John Sheehy wrote:
"acl" wrote groups.com:

On the one hand, I find it hard to believe it's the compression, the
gaps between the levels that are present are smaller than the
theoretical photon noise,

That still posterizes the noise and signal a little bit. You're not
likely to see it with any normal tonal curve; you really need to increase
the contrast quite a bit, and you will see it. For example, I remember
shooting in extreme fog a couple of years ago, where I used +2 EC with my
20D, at ISO 400, and raised the effective blackpoint such that the dark
parts of the Robins approached black. It brought up a bit of noise that
would not normally be seen, with any exposure compensation level, while
black was still anchored at black. Same with taking pictures of things
reflected in glass over a white background, if you try to restore black
in the processing.

Well yes, that is what I was thinking too (ie that posterising the
noise could cause problems under extreme adjustments), but didn't
actually see anything the couple of times I tried (by shooting in forg
and moving the black and white points). I also played a bit with
compressed and uncompressed raw files but could not see anything so
far. Maybe I was not extreme enough.


so basically the extra tonal resolution of
uncompressed raw just records noise more accurately [and since you
can't really see shot noise in reasonably high-key areas, that tells
you it's irrelevant resolution anyway]. On the other hand, who knows?
Maybe there is some indirect effect.

Recording noise better is a good thing, and the same conditions record
signal better as well (and allows the brain and algorithms to separate
them better, as well).

In this particular case, it is only likely to be seen in extreme
blackpointing, or perhaps extreme sharpening.

Yes, obviously you'd expect to see a difference under conditions that
exaggerate small differences, ie extreme tonal stretching or
sharpening (which is local tonal manipulation, after all). But I
didn't.

Well I'll try to play with Paul's example and see what happens
(unfortunately I just remembered I have an early plane to catch
tomorrow so it'll have to wait a bit).