Raw conversion ..

"lebouef" wrote in
:

The raw data is rendered as an image in the video buffer, nothing more
and no different than any other screen image regardless of content.
Raw converters apply a modicum of processing to an image before
opening it or there would be no viewable content, just the 1s and 0s
of digital data. There is no disputing that a Nikon NEF image looks
different opened in Nikon NX compared to the Adobe converter. The
differences reflect aesthetic decisions made by the
programmers/publishers of the program. If you like the way NX opens
images it may shorten your workflow; you may prefer the relative
paucity of processing that the Adobe converter applies. You cannot
save to the raw format from an image processing program in the raw
format. NX and the Adobe converter merely save instructions on how to
open the raw image to its previous but reversible state.



OK this is more or less what I expect but there are a couple of conventions
that need to be fullfilled ... ie de demosaic of the raw needs to generate
"something" that is compatible with the normal representation standard in
de computer hardware ergo IMHO there is at least some need for an RGB pixel
representation in memory. I do not think that it is mapped to video memory
for the simple reason that this is done by the video driver and at this
level the driver needs some prechewed data that it can understand ...

Mike Russell wrote in
:

On 28 Sep 2009 21:14:07 GMT, imbsysop wrote:

I have been wondering for some time..

Is there any info to what type of picture "format" raw is converted
in its intermediate state ie before the user makes a decision to save
it under the form of a known picture format? (jpg, tiff etc)
What I mean is .. raw = converted(=demosaic)= picture in PP program
"memory" .. so what picture "format" is this picture in?
TIA

From what I've picked up, Adobe software, including Lightroom and
Camera Raw, process raw images internally in 16 bit linear gamma, with
the ProPhoto RGB primaries. The actual layout of the pixel data in
memory is either interleaved channel data, with the color data for
each pixel stored side by side, or with each color channel stored
consecutively in memory in its own buffer.

Thank you very much that sheds quite some light on the "raw" picture data
"as seen" !

eNo wrote in
:

On Sep 28, 2:14*pm, imbsysop wrote:
I have been wondering for some time..

Is there any info to what type of picture "format" raw is converted
in it
s
intermediate state ie before the user makes a decision to save it
under t
he
form of a known picture format? (jpg, tiff etc)
What I mean is .. raw = converted(=demosaic)= picture in PP progr
am
"memory" .. so what picture "format" is this picture in?
TIA

I'm assuming you mean, what is the format being held in memory for
display. I'll speak for the Windows OS, for which I wrote a couple of
graphics applications (including a JPG viewer) some time ago.
Essentially, the image must be converted and held in memory in the
Windows API (GDI) Bitmap (BMP) format.

yep indeed ... are you sure about the BMP format? This seems the most
restrictive in terms of color tonalities and format flexibility ?


I haven't handled 16-bit per channel data, as that post-dates my
WinAPI programming days, but I imagine these days the Windows API
contains all the goodies for color handling of such images, as would
be needed to handle 12-bit and 14-bit RAW data. Adobe also has their
own color space, which I've read allows for more nuanced color
manipulation, but again, that extends beyond my nuts-and-bolts
experience. Once you are done editing the image, then the software
would map its internally stored data into whatever format you select
for output. I assume S/W written for other OS's would proceed along
similar lines.

Well I think the 12/14 bits raw matter is not relevant at this level
anymore in so far that the demosaic/conversion has been done before it is
even possible to "see" anything that looks like a real picture .. but is
the conversion outcome a 16 bit format aka 16bit x3 color depth? I guess
so.

So does it make sense to suppose that initially this intermediate format
is "proprietary" to the used program and highest resolution, color depth?

Thanks everyone for your input. The Q was raised in some other (foreign)
forum where people claimed they applied noise reduction to the raw
file/data ..

imbsysop wrote:
"lebouef" wrote in
:

The raw data is rendered as an image in the video buffer, nothing more

A converting program does not need to render anything in the video buffer.



OK this is more or less what I expect but there are a couple of conventions
that need to be fullfilled ... ie de demosaic of the raw needs to generate
"something" that is compatible with the normal representation standard in
de computer hardware ergo IMHO there is at least some need for an RGB pixel
representation in memory. I do not think that it is mapped to video memory
for the simple reason that this is done by the video driver and at this
level the driver needs some prechewed data that it can understand ...

It only needs to be in a format that the converting program understands
and the details of that format do not have to resemble anything but what
was convenient to the programmer of the converting program.

Input(raw) --- Converter [memory space] --- Output(TIF*)

*DNG, or JPG....

For that matter, you can download dcraw.c source code and see how that
converter does it.

On Tue, 29 Sep 2009 07:30:51 -0700, John McWilliams
wrote:

Eric Stevens wrote:
On Mon, 28 Sep 2009 18:00:46 -0700, John McWilliams
wrote:

Matt Clara wrote:
"Wolfgang Weisselberg" wrote in message
...
Savageduck wrote:

In most cases you should end up working in TIF.
Why?
Because it's lossless, capable of handling 16 bit files, preserves
layers and other effects offered by photoshop, et al, allows a couple
forms of lossless compression, none of which a noob needs to know in
order to know which format will best preserve his or her electronic
image file.

Yes, TIFF is tops if you need layers, but keeping the RAW file is the
most complete archival source of a digital image. And it's way smaller
than a TIFF.

But a RAW format does not have and cannot preserve layers.

Which is one reason I wrote what I did. There's not even an inference
that RAW can have layers.

TIFFs have somehow got into the conversation. I was just making an
essential difference clear for those who might think its a toss-up as
to whether to use TIFF of RAW.



Eric Stevens

Eric Stevens wrote:
On Tue, 29 Sep 2009 07:30:51 -0700, John McWilliams
wrote:

Eric Stevens wrote:
On Mon, 28 Sep 2009 18:00:46 -0700, John McWilliams
wrote:

Matt Clara wrote:
"Wolfgang Weisselberg" wrote in message
...
Savageduck wrote:

In most cases you should end up working in TIF.
Why?
Because it's lossless, capable of handling 16 bit files, preserves
layers and other effects offered by photoshop, et al, allows a couple
forms of lossless compression, none of which a noob needs to know in
order to know which format will best preserve his or her electronic
image file.
Yes, TIFF is tops if you need layers, but keeping the RAW file is the
most complete archival source of a digital image. And it's way smaller
than a TIFF.
But a RAW format does not have and cannot preserve layers.
Which is one reason I wrote what I did. There's not even an inference
that RAW can have layers.

TIFFs have somehow got into the conversation. I was just making an
essential difference clear for those who might think its a toss-up as
to whether to use TIFF of RAW.

The whole notion of what output format to use is meaningless to the OP's
question. (Not that that was terribly meaningful).

You can also conserve things like layers in psd format (from photoshop).

Personally I convert to DNG for master storage. And of course since the
notion of layers is pretty useless for a raw image that doesn't matter
either (and now someone will say that DNG encapsulates a TIF ... and it
still doesn't matter 'cause all I care about is the raw image inside...)

Eric Stevens wrote:
On Tue, 29 Sep 2009 07:30:51 -0700, John McWilliams
wrote:

Eric Stevens wrote:
On Mon, 28 Sep 2009 18:00:46 -0700, John McWilliams
wrote:

Matt Clara wrote:
"Wolfgang Weisselberg" wrote in message
...
Savageduck wrote:

In most cases you should end up working in TIF.
Why?
Because it's lossless, capable of handling 16 bit files, preserves
layers and other effects offered by photoshop, et al, allows a couple
forms of lossless compression, none of which a noob needs to know in
order to know which format will best preserve his or her electronic
image file.
Yes, TIFF is tops if you need layers, but keeping the RAW file is the
most complete archival source of a digital image. And it's way smaller
than a TIFF.
But a RAW format does not have and cannot preserve layers.
Which is one reason I wrote what I did. There's not even an inference
that RAW can have layers.

TIFFs have somehow got into the conversation. I was just making an
essential difference clear for those who might think its a toss-up as
to whether to use TIFF of RAW.


The whole notion of what output format to use is meaningless to the OP's
question. (Not that that was terribly meaningful).

You can also conserve things like layers in psd format (from photoshop).

Personally I convert to DNG for master storage. And of course since the
notion of layers is pretty useless for a raw image that doesn't matter
either (and now someone will say that DNG encapsulates a TIF ... and it
still doesn't matter 'cause all I care about is the raw image inside...)

"nospam" wrote in message
...
In article , Matt Clara
wrote:

In most cases you should end up working in TIF.

Why?

Because it's lossless, capable of handling 16 bit files, preserves layers
and other effects offered by photoshop, et al, allows a couple forms of
lossless compression, none of which a noob needs to know in order to know
which format will best preserve his or her electronic image file.

the photoshop format itself does all that and more.

plus, many apps these days use a non-destructive workflow directly on
the raw file so there isn't a tif or psd to save.


You can only do so much in those programs, though, and if you want to do
more, then tiff's the way to go.

--
www.mattclara.com

On 2009-09-29 18:53:36 -0700, "Matt Clara" said:

"nospam" wrote in message
...
In article , Matt Clara
wrote:

In most cases you should end up working in TIF.

Why?

Because it's lossless, capable of handling 16 bit files, preserves layers
and other effects offered by photoshop, et al, allows a couple forms of
lossless compression, none of which a noob needs to know in order to know
which format will best preserve his or her electronic image file.

the photoshop format itself does all that and more.

plus, many apps these days use a non-destructive workflow directly on
the raw file so there isn't a tif or psd to save.


You can only do so much in those programs, though, and if you want to
do more, then tiff's the way to go.

Which is why, once you have made LR adjustments, you can edit an
adjusted copy in CS(or any other photo-edit of your choice). There you
can work with layers to your heart's content, with the final product, a
TIF, ending up stacked with the original DNG in LR.
--
Regards,

Savageduck

"imbsysop" wrote in message
...
eNo wrote in
:

On Sep 28, 2:14 pm, imbsysop wrote:
I have been wondering for some time..

Is there any info to what type of picture "format" raw is converted
in it
s
intermediate state ie before the user makes a decision to save it
under t
he
form of a known picture format? (jpg, tiff etc)
What I mean is .. raw = converted(=demosaic)= picture in PP progr
am
"memory" .. so what picture "format" is this picture in?
TIA

I'm assuming you mean, what is the format being held in memory for
display. I'll speak for the Windows OS, for which I wrote a couple of
graphics applications (including a JPG viewer) some time ago.
Essentially, the image must be converted and held in memory in the
Windows API (GDI) Bitmap (BMP) format.

yep indeed ... are you sure about the BMP format? This seems the most
restrictive in terms of color tonalities and format flexibility ?


I haven't handled 16-bit per channel data, as that post-dates my
WinAPI programming days, but I imagine these days the Windows API
contains all the goodies for color handling of such images, as would
be needed to handle 12-bit and 14-bit RAW data. Adobe also has their
own color space, which I've read allows for more nuanced color
manipulation, but again, that extends beyond my nuts-and-bolts
experience. Once you are done editing the image, then the software
would map its internally stored data into whatever format you select
for output. I assume S/W written for other OS's would proceed along
similar lines.

Well I think the 12/14 bits raw matter is not relevant at this level
anymore in so far that the demosaic/conversion has been done before it is
even possible to "see" anything that looks like a real picture .. but is
the conversion outcome a 16 bit format aka 16bit x3 color depth? I guess
so.

So does it make sense to suppose that initially this intermediate format
is "proprietary" to the used program and highest resolution, color depth?

Thanks everyone for your input. The Q was raised in some other (foreign)
forum where people claimed they applied noise reduction to the raw
file/data ..


(To me this is a foreign forum :-) )
DxO (www.dxo.com) claims to apply noise reduction to the raw data.