I've posted a new chart and accompanying article at my site.

This topic is of special interest to low light shooters who shoot raw and always post process.
It's designed to help you make an objective decision about whether it is of value to raise ISO.

Here's a snapshot showing the D7000 and the D3S:



Bill

Visit me at My site

Another interesting graph. But again I am confused on how exactly you arrive at these metrics, and if they truly are objective measures, and not biased towards particular situations (like the concept of PDR is).

I would love to see more detail on the method and theory behind how you generate these data.

Norman

Some of my pictures

As the article states, this is exactly the same data as the PDR, only stated differently.

Visit me at My site

Yes but there is no where that explains how this data is derived and the underlying math. It is some sort of super geek sensor measurements but one big variable that does not seem to be taken into consideration is the firmware that can control the characteristics of IQ coming from a sensor and the effects that changes in firmware can have. Again I ask, who has validated this photographic term you have contrived and use freely in forums as if it is actually an accepted term. I am surprised that Nikonians would have "PDR" in their wiki where one would only expect legitimate and accepted photographic terms. This implies that PDR is a legitimate photgraphic term.

I am rather surprised how people in the forum seem to take the points you put on a graph at face value and without question. Even if we shall suspend reality and pretend that "PDR" is meaningful in some way, what do differences in points actually mean when it comes to differences in image quality when two printed pictures are actually viewed side by side. The D70 appears to be better than the D2X because of points on a graph but what does this difference actually mean when it comes to the printed image???? It is noticeable to the casual viewer or is this difference really seen on a computer screen magnified to 200%? Your points are not put into any real CONTEXT. This is why I feel your graphs are pretty meaningless in the real world of photography.

Mel
An Opinionated Old Curmudgeon from Virginia


Website - www.meltalley.com
Blog - http://blog.meltalley.com
Facebook - www.facebook.com/mel.talley
Twitter - @meltalley

To sum up.....Mel wants to know where you are coming from?

http://www.flickr.com/photos/43019448@N04/

Quite frankly I’m a bit confused by all this. Granted due to circumstances beyond my control that’s a bit easier to do now days than it used to be!!

But being the owner of a D7000 I found your examples interesting and have several questions (one of which is due to your prior PDR chart).

In your article you make this statement, “For those shooting raw it can be useful to know when raising ISO in the camera has little or no advantage over applying digital gain in post processing.” Followed later by this statement, “It’s clear that for the D7000 going beyond ISO 400 has no advantage and that even at ISO 400 the shadow gain is minimal.”
I take that to mean that you are of the opinion that D7000 users should not go above ISO 400, but instead underexpose (set aperture and shutter speed as if you were shooting with higher ISO) and adjust later in post processing.

Also you provide an ideal FX and DX plot, which is a flat line at the zero reference. To me an ideal plot would be something to aim for. In that respect the D7000/D5100 appears to be the only cameras to approach this ideal plot, yet I get a sense you are inferring that other cameras are better at in camera ISO usage than a D7000. Also the D40 appears to go below the zero reference at its highest ISO setting??

My other question is in reference to your PDR plots. I notice the D7000 peaks at around ISO 1000 and sort of flat lines at higher ISO’s. I remember reading a dpreview forum thread that was referenced in a thread in the D7000 forum. I cannot find that dpreview thread, but if I remember correctly there was a woman who did sensor analysis similar to you and stated that the D7000 stopped using analog gain at around ISO 1000. After that it used some form of bit manipulation at higher ISO’s. So in her opinion it was useless to set an ISO greater than 1000 for you could achieve better results yourself in PP. Did you notice anything odd with ISO settings greater than 1000?


Pete

Visit my Nikonians gallery.

What you call photographic dynamic range does not make a lot of difference - you could call it Tom or Mel or Bill and it would still be based on the same facts. PDR does work to the extent that it takes dynamic range and puts into a practical context for photography that allows comparison not only of noise, but the relative impact on resulting photographic images.

If you don't think the data matters - this post is not for you but it is interesting to a number of others. If you have legitimate questions about the methodology, be specific and seek to understand before you criticize the work of others who do have an understanding of the detail.

Bill, I appreciate you sharing this chart. I'm having a hard time understanding why the data varies so much for different camera models. What I think I am seeing is the D3s and D4 are designed for low shadow noise. The D700 and D3 were looking for best balance. And the D7000 and D800 are not optimized for lowest shadow noise, but rather low overall noise and best performance at low ISO levels.

How would you use this data to optimize how you use the new cameras?


Eric Bowles
Nikonians Team
My Gallery
Workshops

Nikonians membership — my most important photographic investment, after the camera

Hi Mel,

>> but there is no where that explains how this data is derived and the underlying math.

You may want to take a look at Bill's Home Page.

Scroll down to the section titled "Sensors and Raw".

Under that section, click on Sensor Analysis Primer". Start reading

I suspect you will learn all you want to know (and perhaps more than you want to know ) about Bill's math and methodology.

Once you have fully digested that, you are certainly entitled to dispute any details of his math or methods. However, in any case where someone's mathematics and methodology is disputed, it isn't good enough to simply criticize it or dismiss it out of hand. You need to prepare an alternative solution that resolves your criticisms. Simply throwing peanuts from the gallery is never credible.

It has been previously pointed out to you that while Bill may have coined his own phrase, his fundamental approach, normalizing to a standard output size, is used by others, including DXO, which is certainly considered to be one of the premier readily available alternative studies of sensor performance.

Given the tremendous amount of effort put forth by Bill over many, many years, and freely shared with us, I think he is entitled to coin his own phrase. To repetitively resort to criticisms of phraseology while ignoring the details lends no credibility whatsoever to your various criticisms, assertions and and complaints.

_________________________________
Neil
Nikonians Team
My Gallery

Eric,

I'm having a hard time understanding why the data varies so much for different camera models.

At the most general level the curves tend to stay flat or rise.
This is a function of how the amplifier(s) and Analog to Digital Converter (ADC) are implemented and the balance of noise between the two (and the photosite itself).
There is some debate about whether a flat curve indicates a better design.
My opinion is that it does not.

The D3S and D7000 are two great examples.

The D3S has extraordinary low light capability. As you raise the ISO on a D3S it really does improve the dynamic range.

The D7000 is a very good low light performer but that performance comes pretty much from great low ISO performance.

How would you use this data to optimize how you use the new cameras?

Some raw shooters use this type of information to choose a lower ISO to shoot at. A D7000 shooter might never go above ISO 400 regardless of the shot and know that they can "push" in post processing without really losing anything. Some people find this convenient and argue that they are less likely to blow highlights.
On the other hand, with the D3S you cannot get away with "pushing" afterwards until ISO 6400 without losing shadow dynamic range.

I don't have a use for this data in my day-to-day shooting (yes, I do take photos )
But I have quantified something in a way that is useful to those who do care.
Prior to this those people either went with their "gut" or made estimates based on the read noise; neither is accurate.

There are also interesting technical features to the curves.
For example, the rise in the D3S at ISO 51200 would seem to indicate the positive effect of some noise reduction.

Regards,
Bill

Visit me at My site

Pete,

I take that to mean that you are of the opinion that D7000 users should not go above ISO 400, but instead underexpose (set aperture and shutter speed as if you were shooting with higher ISO) and adjust later in post processing.

Some D7000 raw shooters take that exact approach. It is valid because of the "flat" curve.

you provide an ideal FX and DX plot, which is a flat line at the zero reference. To me an ideal plot would be something to aim for.

It's true that the idea sensor would have a flat line.
But it's not necessarily true that a non-ideal sensor is better if the curve for that sensor is "flat".

the D40 appears to go below the zero reference at its highest ISO setting??

Yes, and I used to think it was bad data but have verified this just yesterday.
Some cameras exhibit strange behavior at the "Hi" ISO values.
I suspect these are side effects of how digital scaling and high ISO noise reduction interact for those cameras.
Note for example that the D90 trails off too, whereas the D3S has a distinct "bump".

I notice the D7000 peaks at around ISO 1000 and sort of flat lines at higher ISO’s. I remember reading a dpreview forum thread that was referenced in a thread in the D7000 forum. I cannot find that dpreview thread, but if I remember correctly there was a woman who did sensor analysis similar to you and stated that the D7000 stopped using analog gain at around ISO 1000. After that it used some form of bit manipulation at higher ISO’s. So in her opinion it was useless to set an ISO greater than 1000 for you could achieve better results yourself in PP. Did you notice anything odd with ISO settings greater than 1000?

This is not unique to the D7000 but the D7000 is a good example.
(You refer to Marianne Oelund, a terrific researcher and photographer.)
The effect is easier to see on the Read Noise in ADUs Chart
These charts are great for determining the actual analog gain range.

Regards,
Bill

Visit me at My site

This chart and the peculiar behavior of the D7000 has been discussed in the context of "all or none" (should the camera never be shot above ISO 400).

My take on this chart is that it quashes a "conventional wisdom" that was drilled into me since the first day I started researching my then new D70 back in 2004.

That wisdom said "NEVER, NEVER, NEVER UNDEREXPOSE A HIGH ISO IMAGE!"

And I shouted that because it has been shouted at me all these years

What the chart suggests, at least in the somewhat extreme case of the D7000, is that that is not always true and the real truth is more complex. For example, as Bill points out, in a case of difficult highlights, which is very typical of night and low light shooting, just a middle of the road approach would be most helpful.

For example, underexposing a half stop to a stop in the case where those nasty "blinkies" are in bad places when "properly exposed".

While I've always understood that that conventional wisdom was inherently less than 100% correct, I never had a precise way to thread that needle.

Personally, if I had a D7000 I would not top end it at ISO 400 for a general routine, simply because I have a lot of trouble assessing image reviews that are 2 or 3 stops underexposed . Exposure aside, in that case I wouldn't even know what I'm looking at. It'd be nearly a black screen .

But I could study those charts, for any camera, and come to an understanding where there may be a totally (or nearly) free lunch in terms of solving highlight problems... in a more judicious manner. Higher than otherwise necessary ISO's might even make sense in some situations as an extended dynamic range tool (with whatever noise compromises go along with it).

_________________________________
Neil
Nikonians Team
My Gallery

Just as a follow up afterthought...

While, in the case of the D7000 for example, noise would not be a problem, I would think that posterization of shadow detail might be a problem, depending on the camera setup.

As a simple and extreme example, with an 8 bit image, we might have luminosity values of 1, 2 and 3. If we push 2 stops we end up with 4, 8 and 12, with nothing in between. If we push 3 stops it gets progressively (and exponentially!) worse. That would be bad, but then again you said this only applies to raw shooters .

Shooting 12 bit mode would significantly improve on that of course, and I would think 14 bit mode would be more or less mandatory to avoid this problem as much as possible. But having never seriously gotten out of the mind set I mentioned previously, or gone about it in a more methodical and directed way, I'm throwing it out for discussion and consideration before I start doing 14 to 8 bit conversions in my head .

_________________________________
Neil
Nikonians Team
My Gallery

Neil,

For those interesting in taking advantage of this characteristic it would be really nice if the firmware allowed separate ISO settings for NEF and JPG.
It would be cool to shoot at ISO 400 and get an ISO 400 NEF without blown highlights but get a JPG (and see it on the camera back) that was a different ISO; even ISO Auto!

Regards,
Bill

Visit me at My site

Neil,

That's an interesting thought.

It's probably not a practical issue since at most print sizes and viewing distances a lot of photosites are being combined to form one value that we can distinguish.

We're relying on that fact that statistically the Signal and Noise (and SNR) are affected the same way regardless of whether the multiplication (gain) is analog or digital.

Regards,
Bill

Visit me at My site

Hi Bill,

I wasn't thinking about noise at all- strictly posterization. And not just out of camera but after further torturing, after being raised up to 4 stops.

I've managed to posterize 14 bit sunset and sunrise skies, and that was with a normal exposure. I'm just thinking it would be worse because the effect of doing all this in the digital domain is to lop off some bits. I haven't figured out how many based on various scenarios. You would probably figure it in a heartbeat .

_________________________________
Neil
Nikonians Team
My Gallery

>Neil,
>
>For those interesting in taking advantage of this
>characteristic it would be really nice if the firmware allowed
>separate ISO settings for NEF and JPG...

In a dynamic lighting situation I'm not sure I could keep track of all that . Whatever was going on with the raw file would be totally out of sight and mind. I have enough trouble keeping track of one exposure setting!

It's an interesting idea and would better implement raw+jpg using that strategy.

_________________________________
Neil
Nikonians Team
My Gallery

Neil,

Yes, that is a slightly different topic.
Have you tried torturing in 16-bits? I think it helps.
(14-bit -> 16-bit -> torture -> (14 or 8)-bit )


Bill

Visit me at My site

Thanks for the link

(bclaff- one recommendation I would have for your site is to put "home" links on more of your pages... if I click directly through to say the PDR graph, I have no easy one-click way to get to your homepage from which to look at other graphs/articles)

I've looked through more of the technical background portions and either I'm just not understanding, or need to spend more time, but I would like more clarification on the following:

-What exactly is the unit "ADU"? This is used in many of the PTC graphs, etc. Couldn't find a definition but maybe I missed it.

-Maybe I missed it, but I don't see where the PTC is generated. I only found that it is determined by running the NEFUtil program. As the PDR calculation seems to be (?) based on the PTC, I still can't tell how it is measured (and thus cannot answer my question below).

-I'm still confused on how the selected values of CoC and print size affect the PDR measurement. Can you produce graphs that use different CoC and/or print size values?

My concern is that choosing different CoC or print size "standards" will affect the results for each sensor differently. Which ultimately means that the listed PDR values and comparisons are not valid for any situation except prints made at the "standard" size. How does the data change if you choose a "standard" print of 4x6? 40x60? Or going to extremes, a small print size of 1" x 1", or a large print size of 1,000" x 1,000"?

As I (possibly incorrectly) understand it now, the concept of PDR is meant to normalize out the effects of variable pixel pitch between sensors. But, if this is indeed true, would not the effective PDR simply be a function of the sensor efficiency? Which could be loosely correlated to the "generation" of sensor technology used? So a D80-generation sensor of any pixel pitch will perform pretty much the same in terms of PDR, since the overall "effectiveness" of that sensor was limited to the technology of the era. Newer generation sensors would again all perform "roughly the same", but better than the D80-era sensors because they have improved efficiency. Therefore, comparing sensors of the same generation (D4 vs. D800, for example) is ultimately meaningless because the PDR is going to be about the same?

Maybe it would be cool to have a new interactive graph that shows PDR but allows you to change the standard print size or CoC as well.

Norman

Some of my pictures

Norman

I'm sure Bill will weigh in, but the CofC and Print size question are pretty easy.

CofC is most relevant at the low end in that if a print gets small enough, you really don't see any noise. And if it gets big enough, you see noise in every image - or at least enough pixelation and interpolation that it doesn't matter. With upsizing routines the high end is a very large print since even a 12 mp camera can render a nice 20 x 30 print.

The key is that whatever standardization you use, the relative performance of the sensors will not change. It doesn't matter whether you are looking at 8x12 or 16 x 24 - the relative order of performance will be the same. The common methodology is to standardize around a common print size - any print size will do - to be able to compare cameras with different sized sensors.


Eric Bowles
Nikonians Team
My Gallery
Workshops

Nikonians membership — my most important photographic investment, after the camera

Norman,

Great feedback and questions. I may not have all the answers in one post.

ADU is an Analog to Digital Unit Also known as a Digital Number (DN).
It's the number you get from the Analog to Digital Converter (ADC).
For a 14-bit ADC that's an integer from 0 to 16,383.

PTC is a Photon Transfer Curve. It is essentially a plot of Signal on the x-axis and Noise on the y-axis. Various flavors use log axises or substitute Noise with an equivalent value like variance or Signal to Noise Ratio (SNR). Regardless, the chart conveys the same information.
When NefUtil is doing a PDR it builds a PTC internally and then searches for the appropriate region.

The CoC used is the "standard" 0.022mm for DX, 0.033mm for FX, etc.
It's roughly equivalent to an 8" wide print held at arms length.
(Or 16" at twice arms length, etc.)

Relatively speaking, in the "old" days, sensor performance was closer together. As technology gets more complicated performance in each generate is actually drifting apart. Paradoxically, the more sophisticated and complicated the sensors, the more they can differ from one another!

For the most part, but not always, varying CoC would simply move all the curves up/down by about equal amounts.
This is why PDR looks so close to DxOMark Landscape DR even through the criteria are very different.

Regards,
Bill

Visit me at My site