If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below. |
|
|
Thread Tools | Display Modes |
#1
|
|||
|
|||
'extra' lossless compression for camera raw images
I just finished writing a small tool (sgraw) which can losslessly compress camera raw images, usually between 20-to-60%. (Regulars here might recall I have mentioned this 'possibility' earlier, I finally put in some hours to actually build a working demo :-) http://www.sachingarg.com/compression/sgraw/ That would be savings between 1GB to 3GB for every 5GB of data. Even when camera has already compressed the raw image (which isn't always the case), its still possible to squeeze out extra compression as cameras have to use simple algorithms for speed. Compression is totally lossless, we get back the exact file we start with (just like with zip), so both pixel information and meta-data are perfectly restored. This is just the first quick hack version (and supports all nikon and fuji cameras), so it obviously wouldn't make sense to actually use it for backups etc, but if there is enough interest I can build it further to support other raw formats, along with improved compression and speed. Your thoughts on this? Sachin Garg [India] www.sachingarg.com | www.imagecompression.info |
#2
|
|||
|
|||
'extra' lossless compression for camera raw images
"Sachin Garg" wrote:
I just finished writing a small tool (sgraw) which can losslessly compress camera raw images, usually between 20-to-60%. (Regulars here might recall I have mentioned this 'possibility' earlier, I finally put in some hours to actually build a working demo :-) http://www.sachingarg.com/compression/sgraw/ That would be savings between 1GB to 3GB for every 5GB of data. Even when camera has already compressed the raw image (which isn't always the case), its still possible to squeeze out extra compression as cameras have to use simple algorithms for speed. Compression is totally lossless, we get back the exact file we start with (just like with zip), so both pixel information and meta-data are perfectly restored. This is just the first quick hack version (and supports all nikon and fuji cameras), so it obviously wouldn't make sense to actually use it for backups etc, but if there is enough interest I can build it further to support other raw formats, along with improved compression and speed. Your thoughts on this? You should patent it. |
#3
|
|||
|
|||
'extra' lossless compression for camera raw images
"Sachin Garg" wrote in
: I just finished writing a small tool (sgraw) which can losslessly compress camera raw images, usually between 20-to-60%. (Regulars here might recall I have mentioned this 'possibility' earlier, I finally put in some hours to actually build a working demo :-) http://www.sachingarg.com/compression/sgraw/ That would be savings between 1GB to 3GB for every 5GB of data. Do you have any experience with freeware tools [preferably commandline] which do Lossy Compression of jpegs? The tool by JDO works well but it is proprietary and not freeware. It can regularly wring out 30% savings in byte size and still retain the "viewability" as a jpg file. Of course quality will be lost depending on the quality of the input.. |
#4
|
|||
|
|||
'extra' lossless compression for camera raw images
The RAW file is already pretty low in file size compared to the PSD and TIFF
file size. Harddrive space is also pretty low in cost too. So, would I use it? Unfortunately no. "Sachin Garg" wrote in message ... I just finished writing a small tool (sgraw) which can losslessly compress camera raw images, usually between 20-to-60%. (Regulars here might recall I have mentioned this 'possibility' earlier, I finally put in some hours to actually build a working demo :-) http://www.sachingarg.com/compression/sgraw/ That would be savings between 1GB to 3GB for every 5GB of data. Even when camera has already compressed the raw image (which isn't always the case), its still possible to squeeze out extra compression as cameras have to use simple algorithms for speed. Compression is totally lossless, we get back the exact file we start with (just like with zip), so both pixel information and meta-data are perfectly restored. This is just the first quick hack version (and supports all nikon and fuji cameras), so it obviously wouldn't make sense to actually use it for backups etc, but if there is enough interest I can build it further to support other raw formats, along with improved compression and speed. Your thoughts on this? Sachin Garg [India] www.sachingarg.com | www.imagecompression.info |
#5
|
|||
|
|||
'extra' lossless compression for camera raw images
On Wed, 19 Mar 2008 01:28:47 +0530, "Sachin Garg"
wrote: I just finished writing a small tool (sgraw) which can losslessly compress camera raw images, usually between 20-to-60%. (Regulars here might recall I have mentioned this 'possibility' earlier, I finally put in some hours to actually build a working demo :-) http://www.sachingarg.com/compression/sgraw/ That would be savings between 1GB to 3GB for every 5GB of data. Even when camera has already compressed the raw image (which isn't always the case), its still possible to squeeze out extra compression as cameras have to use simple algorithms for speed. How much more can you squeeze out of a compressed raw image? When I shoot raw, I use Nikon compressed raw. I just tried bzip2 on a few files. I get about 9% further compression. I don't know exactly what compression algorithm Nikon uses but I doubt it's much more complicated than a requantization of the data to reduce the bits per pixel. Nothing near as complicated as what bzip2 is doing. 9% savings isn't worth the effort for me. But 20-60% definitely would be if you get that with compressed raw. Steve |
#6
|
|||
|
|||
'extra' lossless compression for camera raw images
Sachin Garg wrote:
I just finished writing a small tool (sgraw) which can losslessly compress camera raw images, usually between 20-to-60%. (Regulars here might recall I have mentioned this 'possibility' earlier, I finally put in some hours to actually build a working demo :-) http://www.sachingarg.com/compression/sgraw/ That would be savings between 1GB to 3GB for every 5GB of data. Even when camera has already compressed the raw image (which isn't always the case), its still possible to squeeze out extra compression as cameras have to use simple algorithms for speed. Compression is totally lossless, we get back the exact file we start with (just like with zip), so both pixel information and meta-data are perfectly restored. This is just the first quick hack version (and supports all nikon and fuji cameras), so it obviously wouldn't make sense to actually use it for backups etc, but if there is enough interest I can build it further to support other raw formats, along with improved compression and speed. Your thoughts on this? For GP use? No biggie. Disk space is cheap. If it can be reduced to a h/w implementation (by the camera co's) that is more energy efficient (and faster), then it would be a great thing to improve in camera storage. Once you offload from the camera (flash) to disk, the memory is not that important (to me). Disk space is cheap. Cheers, Alan -- -- r.p.e.35mm user resource: http://www.aliasimages.com/rpe35mmur.htm -- r.p.d.slr-systems: http://www.aliasimages.com/rpdslrsysur.htm -- [SI] gallery & rulz: http://www.pbase.com/shootin -- e-meil: Remove FreeLunch. |
#7
|
|||
|
|||
'extra' lossless compression for camera raw images
"Tony Polson" wrote in message ... "Sachin Garg" wrote: I just finished writing a small tool (sgraw) which can losslessly compress camera raw images, usually between 20-to-60%. (Regulars here might recall I have mentioned this 'possibility' earlier, I finally put in some hours to actually build a working demo :-) http://www.sachingarg.com/compression/sgraw/ That would be savings between 1GB to 3GB for every 5GB of data. [] You should patent it. Thanks for that tip. Like all research my work is also based on hard work from many other researchers, I will explore if there are any patentable parts. Sachin Garg [India] www.sachingarg.com | www.imagecompression.info |
#8
|
|||
|
|||
'extra' lossless compression for camera raw images
"Mike Mills" wrote in message .78... "Sachin Garg" wrote in : I just finished writing a small tool (sgraw) which can losslessly compress camera raw images, usually between 20-to-60%. (Regulars here might recall I have mentioned this 'possibility' earlier, I finally put in some hours to actually build a working demo :-) http://www.sachingarg.com/compression/sgraw/ That would be savings between 1GB to 3GB for every 5GB of data. Do you have any experience with freeware tools [preferably commandline] which do Lossy Compression of jpegs? If you want lossy compression on commandline, you can try the opensource options like "libjpeg" or "libpnm". Even if you don't want to play with sourcecode, you should be able to find its (free) executables for any popular platform. On the other hand, if you want further 'lossless' compression of Jpeg files, look for "jpegoptim" (also opensource). Many popular image editors like photoshop and irfanview etc also allow the jpegoptim optimizations in batch mode (if you dont want to use a commandline tool). The tool by JDO works well but it is proprietary and not freeware. It can regularly wring out 30% savings in byte size and still retain the "viewability" as a jpg file. Of course quality will be lost depending on the quality of the input.. You probably already know that moderate jpeg loss is less of a problem when 'viewing', but it hurts when editing or priniting after the fact. Dont do 'too much' of it :-) Sachin Garg [India] www.sachingarg.com | www.imagecompression.info |
#9
|
|||
|
|||
'extra' lossless compression for camera raw images
"Ali" wrote in message ... "Sachin Garg" wrote in message ... I just finished writing a small tool (sgraw) which can losslessly compress camera raw images, usually between 20-to-60%. (Regulars here might recall I have mentioned this 'possibility' earlier, I finally put in some hours to actually build a working demo :-) http://www.sachingarg.com/compression/sgraw/ That would be savings between 1GB to 3GB for every 5GB of data. The RAW file is already pretty low in file size compared to the PSD and TIFF file size. Yep, that is correct. Similar savings are 'technically possible' on TIFF and PSD files too, but that would be another story. Harddrive space is also pretty low in cost too. So, would I use it? Unfortunately no. ok Sachin Garg [India] www.sachingarg.com | www.imagecompression.info |
#10
|
|||
|
|||
'extra' lossless compression for camera raw images
"Steve" wrote in message ... On Wed, 19 Mar 2008 01:28:47 +0530, "Sachin Garg" wrote: I just finished writing a small tool (sgraw) which can losslessly compress camera raw images, usually between 20-to-60%. (Regulars here might recall I have mentioned this 'possibility' earlier, I finally put in some hours to actually build a working demo :-) http://www.sachingarg.com/compression/sgraw/ That would be savings between 1GB to 3GB for every 5GB of data. Even when camera has already compressed the raw image (which isn't always the case), its still possible to squeeze out extra compression as cameras have to use simple algorithms for speed. How much more can you squeeze out of a compressed raw image? When I shoot raw, I use Nikon compressed raw. I just tried bzip2 on a few files. I get about 9% further compression. I don't know exactly what compression algorithm Nikon uses but I doubt it's much more complicated than a requantization of the data to reduce the bits per pixel. Nothing near as complicated as what bzip2 is doing. While Nikon does uses the requantization you mention, they also compress those quantized pixels further using lossless encoding. And yes, its still a 'simple' algorithm not as complicated as bzip2, but bzip2 type methods (based on bwt) can't perform good on 2D data like images. 9% savings isn't worth the effort for me. But 20-60% definitely would be if you get that with compressed raw. You should be able to get atleast 20-30% even on compressed raw files using sgraw. If you try the tool on your images, I would love to see what results you get. In case you get anything less than that it will mean I can further improve it for your camera's raw files. Sachin Garg [India] www.sachingarg.com | www.imagecompression.info |
Thread Tools | |
Display Modes | |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Extra! Extra! Suicidal Terrorists Attack Cheney Near Chicago's O'Hare | ASAAR | Digital Photography | 0 | April 15th 07 07:21 PM |
Help with compression research (Need sample images) | Sachin Garg | Digital Photography | 8 | January 31st 07 01:43 PM |
question about using jpegtran for lossless compression of jpegs | [email protected] | Digital Photography | 4 | October 24th 06 10:55 AM |
(Lossy) compression of 16bit images | Alfred Molon | Digital Photography | 3 | February 3rd 06 07:52 AM |
Digital Photography Tip #2: Avoid using too much in-camera compression | Gary Hendricks | Digital Photography | 6 | December 5th 04 12:45 AM |