"Raw" file issues?

wrote:

The native RAW files don't have to "handle" it, either.

Once again, nitwit: I have data that hints that Canon has a special
black-level estimator for long exposure images.

Good for you.

This estimator, if it exists, would be built into Canon CR2 file
decoders.

None of this is known to DNG. (I am still awaiting a DNG spec
citation.)

Then I'm awaiting an explanation of why DNG should have to know about it.
Given that it *already works* in DNG *right now*, I think it's pretty
clear that it doesn't, any more than it has to know about every other
camera that comes out.

I'm currently processing my camera's RAW files after converting to DNG.
My camera didn't exist at the time the version of Camera Raw I'm using
was written. Yet, somehow, it works perfectly. How do you suppose that
is? How could DNG possibly "know about" my camera's unique RAW format?

Because it doesn't have to, of course.

Given this context, why should I use DNG over CR2?

You shouldn't. No one is trying to make you. But neither should you
argue against DNG based on faulty understanding of what it is. You
are assuming that something Canon is doing would require special
consideration in the RAW file format; it does not require any such
thing.

If something Canon is doing results in their RAW conversion software
doing a better job than third-party software does, that is a completely
different thing and it has nothing, at all, to do with the RAW file format.

--
Jeremy |

On Fri, 27 May 2005 19:26:50 +0000 (UTC), Ben Rosengart
wrote:

Incidentally, have you ever read about NASA's problems interpreting
their own old data from the '60s?

No, was it anything about how to avoid expensive human fireworks?

....again, I'm not NASA. And they could just as easily have problems
interpreting data from last year. Mainstream is the key here, and they
aren't a good example of that.

I'm not denying that some data might be difficult to read in the
future, especially if it's obscure. I just find it hard to believe
that a file format we are producing literally billions of each year
will suddenly become unaccessible to us. It just makes no sense. Laws,
wars or religion ain't going to stop us from being able to read those
files.

Part of my faith is obviously due to projects like DNG, but the
internet and the way I've seen people who use it attack projects such
as MAME (http://www.mame.net/) give me a great deal of confidence.

--
Owamanga!
http://www.pbase.com/owamanga

Owamanga wrote:

Mainstream files, standard or not, will be able to be read in the
future, weird-arse files are at risk, I agree. So, buy a Canon or a
Nikon, shoot RAW and worry not.

Canon has already dropped support for the RAW files from one of their
cameras, in their new software.

--
Jeremy |

On Fri, 27 May 2005 19:42:11 GMT, Owamanga wrote:

I'm not denying that some data might be difficult to read in the
future, especially if it's obscure. I just find it hard to believe
that a file format we are producing literally billions of each year
will suddenly become unaccessible to us.

Accessibility is relative. I don't expect that current RAW files
of various types will cease to be accessible *period*. But I do
expect that it will gradually become more difficult. I'm not
comfortable with a format unless there are free-as-in-speech programs
that can interpret it. We do have dcraw, which is nice. However,
if the manufacturers wanted to make Dave Coffin's life more difficult,
they could do it, with technology or lawyers. This is a precarious
state of affairs. DNG has the potential to us on firmer ground.

--
Ben Rosengart (212) 741-4400 x215
Sometimes it only makes sense to focus our attention on those
questions that are equal parts trivial and intriguing.
--Josh Micah Marshall

On Fri, 27 May 2005 19:50:27 -0000, Jeremy Nixon
wrote:

Owamanga wrote:

Mainstream files, standard or not, will be able to be read in the
future, weird-arse files are at risk, I agree. So, buy a Canon or a
Nikon, shoot RAW and worry not.

Canon has already dropped support for the RAW files from one of their
cameras, in their new software.

Well, it's still available for download from their webpage.
http://consumer.usa.canon.com/ir/con...m odelid=8294

And there are plenty of other programs that can convert them. The
number is growing, not shrinking.

--
Owamanga!
http://www.pbase.com/owamanga

Owamanga wrote:

And there are plenty of other programs that can convert them.

Yeah, good thing it's possible for third parties to do that, completely
and accurately, eh? It would sure suck if the camera makers decided to
try to prevent that, wouldn't it?

--
Jeremy |

Jeremy Nixon wrote:


Canon has already dropped support for the RAW files from one of their
cameras, in their new software.

If you're going to drop message like that then specifics are in order.

On Fri, 27 May 2005 20:22:17 -0000, Jeremy Nixon
wrote:

Owamanga wrote:

And there are plenty of other programs that can convert them.

Yeah, good thing it's possible for third parties to do that, completely
and accurately, eh? It would sure suck if the camera makers decided to
try to prevent that, wouldn't it?

Indeed, but in the long run people vote with their feet. The Nikon
thing will be interesting to watch - how this all plays out. What I
don't expect is that they cunningly slip in some strong encryption
into my next firmware D70 update... they could, but they won't.

You could extend this argument and suggest that hardware manufacturers
aught to turn over the API and technical docs so we can all write our
own firmware, because god forbid, they one day stop doing firmware
updates for our old fossils.

I'd be nice to have, (the firmware thing) but I can't see it ever
happening.

--
Owamanga!
http://www.pbase.com/owamanga

On Fri, 27 May 2005 16:25:32 -0400, Alan Brownbe
wrote:

Jeremy Nixon wrote:


Canon has already dropped support for the RAW files from one of their
cameras, in their new software.

If you're going to drop message like that then specifics are in order.

If this is accurate, it was probably an oversight. As I've mentioned,
all related software is still available from their website alongside
what appears to be good support of the camera all around.

They even include an invite to get the SDK which would allow you to
write your own interface to the camera, which is better than can be
said of some *other* manufactures....

--
Owamanga!
http://www.pbase.com/owamanga

Owamanga (not-this-bit) wrote:
[snip]
Either DNG is:

1) Simply a container, imbecilically storing the RAW file within
without any comprehension as to its content. Basically no better than
zipping the damn RAW file up.

No. Code can be written that uses the DNG specification to turn a
DNG-conformant file into a recognisable image, without needing to know
anything about the camera concerned. So DNG obviously has the necessary
semantics.

(DNG is a variant of TIFF/EP. So is the typical Raw file format.
Typically, much of the important data has the same meaning in both
formats).

2) A new representation of the data within the RAW file, which
therefore must either understand and re-represent every aspect of the
original RAW file in an open format manner or be considered a lossy
storage method.

No. The essential image data is understand, but often not
re-represented. Some contents of Raw file formats are already likely to
conform to DNG, because they conform the recognised parts of TIFF/EP.
If a Raw format already has the sensor data in the required format, why
re-represent it? (The fact that a file format is proprietary and not
open doesn't mean that "every aspect of the original RAW file" is
closed, not open. Surely the bits that conform to TIFF/EP can be
considered to be open?).

In addition, DNG offers the option of including non-essential private
data that is not dictated by the specification. So it doesn't
understand "every aspect".

3) Some hybrid of the above, meaning it must be significantly larger
than the RAW file originally was.

No. In all cases that I've tested, without embeding the original file,
the DNG version was smaller than the original version. But only when
losslessly compressed. So this is probably just a visible indication of
the relative effectiveness of the lossless compression used by DNG
compared with the compression used in the original format.

(Note that even a D2X NEF file saved by Nikon Capture 4.2.1, while
smaller than that output by the D2X camera, is larger than the DNG
version. I think DNG simply uses a very effective lossless compression
scheme).

Until DNG is so good, that in every case, once a RAW is converted to
DNG we can safely throw the original RAW file away and have lost
nothing, it'll have extremely limited appeal.

No. I don't save the original after converting to DNG. I've been losing
things converting to DNG, such as the lens-model used for the shot. But
those things haven't been important enough to me to do otherwise. Lots
of other people appear to think likewise.

My guess is that DNG will be the format of choice for most Raw shooters
who use CS2 and ACR 3.1. It is so convenient. And with about 25
non-Adobe products supporting DNG, its use won't be confined to
Photoshop / ACR.

Even if Adobe's solution promises to be able to recreate the original
RAW file from the DNG, that's really zero steps better than a zip
file.

No. One method of being able to recreate the original Raw file from the
DNG is to embed the original file in the DNG. Apart from the size
problem, this can be attractive. The file behaves, of course, just like
a DNG file, and can therefore be used in all cases where DNG would be
used. But the original can be extracted with bit-level accuracy at any
time.

Frankly, I haven't a clue whether the "unique black level estimation
for long exposure images" is catered for by the current version of DNG.
Since I don't use Canon, I have no need to think about it. Why couldn't
it be held in DNGPrivateData, or in MakerNote with MakerNoteSafety set
to 1 (safe)? If it can't, then Canon could ask for a new version. That
is why DNG has a good version scheme.

--
Barry Pearson
http://www.barry.pearson.name/photography/
http://www.birdsandanimals.info/