Astro Features??

All things embedded

Astro Features??

Postby Simeon » Tue Feb 19, 2013 2:23 am

For some reason I can't seem to post my reply to this DPreview thread Estimating D7000 dark current so I'll post here:

Of maybe Astro interest is that the Nikon's have three undocumented service modes, that change how the NEF files is created, the first allows the picture to be bigger than normal, thus my d5100 goes from 4928x3264 to 5056x3358, which I think is the black noise area is included.

The second seems to not apply white balance to the embedded image, but I also wonder if the three colour channels are not scaled.

The third is uncompressed NEF files, so not lost data.

Here are five pictures with my lens cap on, 30 seconds ISO 400 at about 22c. In lightroom I turned off all sharping, zero'd black, contrast etc, and boosted exposure to 4.0 and there are 100% zooms of the same-ish area notice blue pixel patterns:

cap_off_off_off.png
cap_off_off_off.png (382.25 KiB) Viewed 3893 times

all three features off, base line picture

cap_on_off_off.png
cap_on_off_off.png (351.68 KiB) Viewed 3893 times

oversize on, others off

cap_off_on_off.png
cap_off_on_off.png (295.07 KiB) Viewed 3893 times

WB? on, others off

cap_on_on_off.png
cap_on_on_off.png (297.48 KiB) Viewed 3893 times

oversized & WB? on, comp lossy

cap_on_on_on.png
cap_on_on_on.png (333.34 KiB) Viewed 3893 times

oversize on, WB? on, no compression (33MB vs 14MB)

Looking at the histograms of the NEF files, the WB? on does shift the colour spectrum, but not as much as the oversized.

The five original files are uploaded in Zip of 5 file (58MB)
Simeon
Core Developer
 
Posts: 2100
Joined: Wed Nov 30, 2011 6:12 am
Location: Christchurch, New Zealand
Been thanked: 398 times

Re: Astro Features??

Postby astronomer » Wed Feb 20, 2013 10:15 am

Great job Simeon!

I'm looking into this detail
The bigger one seems to include the optical black region and overscan to allow black level compensation.

I guess this service mode is used to output the original file for hot spot and dark spot compensation.
User avatar
astronomer
Developer
 
Posts: 272
Joined: Fri Apr 06, 2012 10:06 am
Location: US
Been thanked: 65 times

Re: Astro Features??

Postby astronomer » Wed Feb 20, 2013 10:42 am

OK, after preliminary examination, I could say that the "oversize" mode is linked with median filter (star eater) algorithm during long exposure. Once turn that on, what you get is an unaltered RAW image disregard the exposure time. The typical signature of paired hot spots during long exposure is gone when oversize is turn on.

Actually I would call it overscan mode for an analog in CCD astronomy where you scan the entire frame plus some non-existing area.

Great job! So can you make a similar patch for D7000 so I will be able to test it and give you feed back?

Best regards,
Jack
User avatar
astronomer
Developer
 
Posts: 272
Joined: Fri Apr 06, 2012 10:06 am
Location: US
Been thanked: 65 times

Black Level

Postby astronomer » Thu Feb 21, 2013 1:34 pm

Simeon wrote:For some reason I can't seem to post my reply to this DPreview thread Estimating D7000 dark current so I'll post here:

Of maybe Astro interest is that the Nikon's have three undocumented service modes, that change how the NEF files is created, the first allows the picture to be bigger than normal, thus my d5100 goes from 4928x3264 to 5056x3358, which I think is the black noise area is included.
....


Did you activate this by the Software or did you alter the firmware?
I found there's another service option to estimate the dark current, that might allow you to bypass the black level correction to preserve the full distribution of read noise and any low light detail!
1.jpg
1.jpg (28.83 KiB) Viewed 3669 times
User avatar
astronomer
Developer
 
Posts: 272
Joined: Fri Apr 06, 2012 10:06 am
Location: US
Been thanked: 65 times

Re: Astro Features??

Postby Simeon » Thu Feb 21, 2013 2:00 pm

astronomer wrote:Did you activate this by the Software or did you alter the firmware?
I found there's another service option to estimate the dark current, that might allow you to bypass the black level correction to preserve the full distribution of read noise and any low light detail!


I used that API (external software connected via USB) that that tools is using to do the work, and I've just made a patch for the D7000, that when you change the Jpeg Compression, it "should" turn on/off the Oversampling instead. Just uploading version 1.11 Now.
Simeon
Core Developer
 
Posts: 2100
Joined: Wed Nov 30, 2011 6:12 am
Location: Christchurch, New Zealand
Been thanked: 398 times

Re: Astro Features??

Postby astronomer » Thu Feb 21, 2013 10:48 pm

Simeon wrote:
astronomer wrote:Did you activate this by the Software or did you alter the firmware?
I found there's another service option to estimate the dark current, that might allow you to bypass the black level correction to preserve the full distribution of read noise and any low light detail!


I used that API (external software connected via USB) that that tools is using to do the work, and I've just made a patch for the D7000, that when you change the Jpeg Compression, it "should" turn on/off the Oversampling instead. Just uploading version 1.11 Now.


I've flash the firmware into my D7000. The menu option for JPEG seems to be disabled but I still didn't get the oversized RAW image.
Do I need tether shooting in order to get the service mode?

Edited:
Did you mean an API in one of the dll that service software provided? Do you have a tool to do this?
User avatar
astronomer
Developer
 
Posts: 272
Joined: Fri Apr 06, 2012 10:06 am
Location: US
Been thanked: 65 times

Re: Astro Features??

Postby Simeon » Sat Feb 23, 2013 11:19 pm

astronomer wrote:I've flash the firmware into my D7000. The menu option for JPEG seems to be disabled but I still didn't get the oversized RAW image.
Do I need tether shooting in order to get the service mode?


Grrr, was going away for weekend so didn't test the change on my D5100 first to know it should work for you. Will try getting it working for me, and then post new patch for you.

astronomer wrote:Edited:
Did you mean an API in one of the dll that service software provided? Do you have a tool to do this?
I am send USB commands directly VIA libgphoto2 (altered to do my bidding). But they are the same USB commands that Nikon's DLL will be sending.
Simeon
Core Developer
 
Posts: 2100
Joined: Wed Nov 30, 2011 6:12 am
Location: Christchurch, New Zealand
Been thanked: 398 times

Re: Astro Features??

Postby astronomer » Sun Feb 24, 2013 12:17 pm

Simeon wrote:
astronomer wrote:I've flash the firmware into my D7000. The menu option for JPEG seems to be disabled but I still didn't get the oversized RAW image.
Do I need tether shooting in order to get the service mode?


Grrr, was going away for weekend so didn't test the change on my D5100 first to know it should work for you. Will try getting it working for me, and then post new patch for you.

astronomer wrote:Edited:
Did you mean an API in one of the dll that service software provided? Do you have a tool to do this?
I am send USB commands directly VIA libgphoto2 (altered to do my bidding). But they are the same USB commands that Nikon's DLL will be sending.

Are U saying that only throuth tether shooting can you get oversized raw? Right now i'm using in camera raw shooting and there seemed to be no effect.
User avatar
astronomer
Developer
 
Posts: 272
Joined: Fri Apr 06, 2012 10:06 am
Location: US
Been thanked: 65 times

Re: Astro Features??

Postby Simeon » Sun Feb 24, 2013 12:44 pm

astronomer wrote:Are U saying that only through tether shooting can you get oversized raw? Right now i'm using in camera raw shooting and there seemed to be no effect.


I am saying I can turn the Oversized feature on, when my camera is plugin into my PC via USB, and then it remains on until I power cycle it.

I'm trying to make a software patch that has it turned on/off via menus, this is what I failed to do for you in the software patch. I can ether try make a release where it is always on, or one where you can turn stuff on/off (by replacing un-useful stuff like in camera HDR). I assumed the latter would be more useful.
Simeon
Core Developer
 
Posts: 2100
Joined: Wed Nov 30, 2011 6:12 am
Location: Christchurch, New Zealand
Been thanked: 398 times

Re: Astro Features??

Postby astronomer » Sun Feb 24, 2013 7:08 pm

Simeon wrote:
astronomer wrote:Are U saying that only through tether shooting can you get oversized raw? Right now i'm using in camera raw shooting and there seemed to be no effect.


I am saying I can turn the Oversized feature on, when my camera is plugin into my PC via USB, and then it remains on until I power cycle it.

I'm trying to make a software patch that has it turned on/off via menus, this is what I failed to do for you in the software patch. I can ether try make a release where it is always on, or one where you can turn stuff on/off (by replacing un-useful stuff like in camera HDR). I assumed the latter would be more useful.


That sounds awesome, looking forward to that patch.
I figured out that for amateur astronomers, the least used menu option is the RAW compression. I always set it to lossless. The lossy will discard too much detail if you want hard processing.
User avatar
astronomer
Developer
 
Posts: 272
Joined: Fri Apr 06, 2012 10:06 am
Location: US
Been thanked: 65 times

Next

Return to Firmware

Who is online

Users browsing this forum: No registered users and 2 guests

cron