Understanding Firmware

From Nikon Hacker
Jump to: navigation, search


Most electronic devices nowadays are based on digital chips called microcontrollers, that run a program called firmware. This firmware is responsible for all operations, from initializing the device, checking all peripherials and other components are OK and ready to use, driving menu interface, responding to buttons, and ultimately perform what the device is designed for.

The Nikon DSLRs are such devices, and contain several chips, among which several microcontrollers. Two of them (called A and B in short) perform most of the hi-level logic. Like most advanced device makers, Nikon has foreseen the possibility that their cameras could benefit from improvements or bugfixes. The firmwares for those two microcontrollers are thus stored in semi-permanent memory (flash) that can be rewritten by a specific procedure to "update the firmware" (that is, replace it with another version including desired changes). This is known as "flashing" the camera.

For all cameras that have already received such updates, Nikon makes the updated firmware(s) available as downloadable files on their website. Recent firmwares (A and B) are combined into a single file, which is encrypted, then compressed into an executable with the RAR tool. To get the encrypted combined binary file, simply execute the downloaded file. You then normally put the resulting file on a memory card to flash the camera.

If you want to work on the firmware, instead of flashing the camera, you have first to decrypt that file. The encryption algorithm has been found at the end of 2011 by Simeon Pilgrim and you can find the details starting from this post on his blog. Several tools are now available to decrypt the firmware, one of them being built in the NikonEmulator.

Once decrypted and separated, the A and B firmware are just two binary files that are understood by the camera microcontrollers, but are not (yet) human readable.

The "B" microcontroller

General information

The "B" microcontroller is the one we mainly focus on because it drives the screen and handles user interaction such as menu navigation. It is a Nikon Expeed ASSP and is based on a Fujitsu Milbeaut-4 32-bit microcontroller, including processor core of FR80 family, like the chips numbered 916xx or 91F6xx to be found here.

As such, the firmware that drives the menus and the basic logic of the camera consists of binary machine code understood by that specific family of microcontrollers. That binary code can be converted back to very low-level human-readable code (assembly language). (Note: Nikon didn't write most of the firmware in assembly, but in a higher-level language such as C. Although, as we don't have access to these "sources", all we can do is start from what we have - compiled machine code files - and translate them into assembly language)

To dig into the B firmware code, it is thus highly advised to first carefully read chapters 1 to 3 of the FR Family instruction manual (see reference below). Those 30 pages are really a prerequisite before trying to follow or ask questions about internals of the code. For more advanced understanding of that microcontroller, reading the FR80 Programming manual is advised, because it approaches other aspects of the chip and has a slightly modified instruction set, used in the Expeed.

Fortunately, work has been done to convert the binary format to human readable "text" format known as assembly language. A good deal of the work was made by Kevin Schoedel in his Dfr disassembler, and this disassembler was improved and included in several tools such as the NikonEmulator.

Address map

To the best of our knowledge, the basic address map of the D5100 "B" firmware is as follows :

Address range Size Bus width Description
0x00000000-0x00000FFF 4KB intern I/O registers. They are used to configure the different circuits that live on the chip along with the CPU. All information we find with respect to this I/O map is summarized into this spreadsheet
0x00040000-0x0103FFFF 16MB 16-bit External bus CS0: MCIX flash chip with Firmware (note this area overlaps with BootROM region below!). That firmware contains code (CPU init, OS, libraries, and Nikon-specific code, the entry point being at 0x40000), another interrupt vector, and data (pointer tables, localized strings, icons, JPEG files, ...)
0x000E0000-0x000FFFFF 128KB intern Boot ROM (overlap region). It contains the original interrupt vector as well as a bootloader with "recovery" functionality in case firmware update fails, or which gives control to the firmware code if valid. Flashing the 0xE0000-FFFFF area is skipped by the firmware loader.
0x03000000-0x0300FFFF 64KB 16-bit External bus CS2: ?? seems to be flash
0x30000000-0x3003FFFF 256KB intern same contents as 0x68000000-0x6803FFFF
0x40000000-0x40001FFF 8KB 16-bit External bus CS3: Milbeaut image hardware macros
0x40060000-0x4006FFFF 64KB intern Image processor special registers ??
0x40070000-0x40077FFF 32KB intern Fujitsu DSP Table RAM
0x40170000-0x40177FFF 32KB intern Fujitsu DSP Data RAM
0x401C0000-0x401C00FF 256B intern Fujitsu DSP Core 1 registers
0x401D0000-0x401D00FF 256B intern Fujitsu DSP Core 0 registers
0x68000000-0x6801FFFF 128KB intern Internal Expeed RAM area with variables: first 2KB is system stack (SSP), currentTCB pointer, TCB Table, etc
0x68020000-0x68027FFF 32KB intern Fujitsu DSP instruction RAM window (2 pages)
0x68027FFF-0x6803FFFF 96KB intern Internal Expeed (DSP?) RAM
0x70000000-0x700001FF 512B 16-bit External bus CS1: ???
0x7F000000-0x7F0001FF 512B 32-bit External bus CS4: ??? LiveView frame control registers ? Dual RAM ? (This area do not use RDY access cycle extension).
0x7F010000-0x7F0103FF 1KB 32-bit External bus CS5: ??? LiveView frame buffer ? Dual RAM? (this area do not use RDY access cycle extension).
0x80000000-0x8FFFFFFF 256MB vary Main SDRAM
0x90000000-0x9FFFFFFF 256MB vary same contents as 0x80000000-0x8FFFFFFF
0xA0000000-0xAFFFFFFF 256MB vary SDRAM
0xB0000000-0xBFFFFFFF 256MB vary same contents as 0xA0000000-0xAFFFFFFF
0xC0000000-0xCFFFFFFF 256MB vary same contents as 0x80000000-0x8FFFFFFF
0xCE57DC60-0xCE5E1C5F 0x64000 bytes intern Screen buffer (LCD): luminance values Y (for YCbCr 4-2-2), one byte per pixel
0xCE5E1C60-0xCE645C5F 0x64000 bytes intern Screen buffer (LCD): blue chrominance values Cb (for YCbCr 4-2-2), one byte for two pixels (rows aligned to 640px)
0xCE645C60-0xCE6A9C5F 0x64000 bytes intern Screen buffer (LCD): red chrominance values Cr (for YCbCr 4-2-2), one byte for two pixels (rows aligned to 640px)
0xD0000000-0xDFFFFFFF 256MB vary same contents as 0x80000000-0x8FFFFFFF
0xE0000000-0xEFFFFFFF 256MB vary same contents as 0xA0000000-0xAFFFFFFF
0xF0000000-0xFFFFFFFF 256MB vary same contents as 0xA0000000-0xAFFFFFFF

For overlapped areas following rules apply (according to Fujitsu documentation for example for MB91460 "mask area setting"):

  • "intern" regions are always seen and CS signal is not issued
  • for memory overlapped between two external areas CS of first area is issued and timing parameters of second area are used for access

After analyse of mirror areas, it is clear that this ASSP chip do not evaluate address lines A30 and A28 while accessing internal bus. But it still evaluates them inside FR80 processor core, so access at 0xCxxxxxxx basically access same memory on internal bus as address 0x8xxxxxxx, but without cache.

Fujitsu DSP

Additional Fujitsu FR-V family processor cores on same die acts as DSP. Basic FR-V instruction set (32-bit instruction width) is supported. More likely it is MB93577: dual core DSP and 256 KB internal RAM match findings.

I/O registers overview

Following devices were observed in D5100 firmware "B"

Address range Related Interrupt Interrupt type Description
0x40-0x43 0x10, 0x11, 0x12?, 0x13?, 0x14?, 0x15, 0x16, 0x17 - External interrupt controller ch 0..7
0x44 0x3F - Delay interrupt
0x48-0x4F 0x18 - Reload Timer 16-bit ch 0
0x50-0x58 0x19 - Reload Timer 16-bit ch 1
0x58-0x5F 0x1A - Reload Timer 16-bit ch 2
0x60-0x6F receive/send 0x1C shared UART/SIO ch 0
0x70-0x7F receive/send 0x1D shared UART/SIO ch 1
0x80-0x8F receive/send 0x1E? shared UART/SIO ch 2
0x90-0x9F receive/send 0x1E? shared UART/SIO ch 3
0xA0-0xAF receive/send 0x1E? shared UART/SIO ch 4
0xB0-0xBF receive/send 0x1B shared UART/SIO ch 5
0xC0-0xCF receive/send 0x1B? shared UART/SIO ch 6
0xF0-0xF3 0x1F?, 0x20?, 0x21, 0x22, ?, 0x23, 0x24, ? - External interrupt controller ch 8..15
0x100-0x10F 0x2E shared Reload Timer 32-bit ch 0
0x110-0x11F 0x2E shared Reload Timer 32-bit ch 1
0x120-0x12F 0x2E shared Reload Timer 32-bit ch 2
0x130-0x13F 0x2E shared Reload Timer 32-bit ch 3
0x140-0x14F 0x2E shared Reload Timer 32-bit ch 4
0x150-0x15F 0x2E shared Reload Timer 32-bit ch 5
0x160-0x16F 0x2E shared Reload Timer 32-bit ch 6
0x170-0x17F 0x2E shared Reload Timer 32-bit ch 7
0x180-0x18F 0x2E shared Reload Timer 32-bit ch 8
0x190-0x19F 0x2E shared Reload Timer 32-bit ch 9
0x1A0-0x1AF 0x2E shared Reload Timer 32-bit ch 10
0x1B0-0x1BF 0x2E shared Reload Timer 32-bit ch 11
0x3E0-0x3EF - - Cache controller
0x440-0x46F - - Interrupt controller
0x480-0x483 - - Reset/power control
0x488-0x48F - - Clock division control
0x490-0x497 - - DMA IORR0...IORR7 start interrupt request
0x600-0x617 - - External bus I/F: ASR0-ASR5
0x640-0x657 - - External bus I/F: ACR0-ACR5
0x680-0x697 - - External bus I/F: AWR0-AWR5
0xD00-0xD1F 0x36 - DMA ch 0
0xD20-0xD3F 0x37 - DMA ch 1
0xD40-0xD5F 0x38 - DMA ch 2
0xD60-0xD7F 0x39 - DMA ch 3
0xD80-0xD9F 0x3A - DMA ch 4
0xDA0-0xDBF 0x3B - DMA ch 5
0xDC0-0xDDF 0x3C - DMA ch 6
0xDE0-0xDFF 0x3D - DMA ch 7
0x4000D810, 0x4000D910 0x24 shared  ???
0x4001xxxx 0x2C shared Debayer unit
0x40020000 0x29 shared Resolution converter ch 0
0x40030000 0x28 shared JPEG decoder ch 0
0x4006000A 0x27 - Image Coprocessor Bus Hub
0x40070xxx - - Bus setup registers 0
0x40090000 0x28 shared Huffman encoder/decoder
0x400A0008 0x2C shared  ???
0x400B0006 0x29 shared  ???
0x400F0000 0x29 shared Resolution converter ch 1
0x40100xxx 0x29 shared Image rotator
0x4011C03A 0x2C shared  ???
0x40130000 0x28 shared JPEG decoder channel 1
0x40170xxx - - Bus setup registers 1
0x40180000 0x28 shared Image Transfer Circuit ch 0 and ch 1
0x40190012 0x28 shared  ???
0x401C0078 0x2B shared DSP Core 1
0x401D0078 0x2B shared DSP Core 0
0x401F0000 0x29 shared Resolution converter ch 2
??? 0x2A shared not used
0x50000000-0x50000024 - - Enable chip components registers
0x50000100 - - I/O port 0:
bit5 output =>TX INTF (0=active)
bit4 output =>TX P52 (rising edge=active)
bit3 input SW2 (0 - TFT reversed, 1 - not)
bit2 input USB Vbus (=0 high, =1 low)
bit0 input SW1 (0 - TFT fully open, 1 - not)
0x50000101 - - I/O port 1:
bit4 IMX071 XCE (0=Communication On, 1=Off)
0x50000102 - - I/O port 2:
bit0 output SD card LED (1=on)
0x50000103 - - I/O port 3:
bit6 input LineOut (0=conected, 1=not)
0x50000104 - - I/O port 4:
bit4 input TFT related ?
0x50000105 - - I/O port 5:
0x50000106 - - I/O port 6:
- I/O port 7:
bit6 input <= TX PC3
0x50000108 - - I/O port 8:
bit4 input SW3 (0 - TFT closed, 1 - not)
0x50000109 - - I/O port 9:
0x5000010A - - I/O port A:

- I/O port B:
bit5 input SD card (0 - unplugged, 1 - in)
bit3 input AF status <= TX ?? (0 - not reached, 1 - reached)
0x5000010C - - I/O port C:
bit4 input external microphone (0 - connected, 1 - not)
bit3 output TFT power (0 - on, 1 - off)
0x5000010D - - I/O port D:
bit3:0 outputs TFT backlight illumination level interface
0x50000200-0x5000020D - - I/O ports configuration: bit value 0: input pin, 1:output pin.
0x50000300-0x5000030D - - I/O ports extended configuration ?
0x5C0010xx - - Audio digital input/output ch 0
0x5C0011xx - - Audio digital input/output ch 1
0x5C002000 0x31 - I2C Bus
0x50010034, 0x50010038 0x32 shared  ???
0x63000000 0x2D shared SD Host Controller ch 0
0x6300010C 0x2D shared  ???
0x64000000 0x2D shared SD Host Controller ch 1
0x6400010C 0x2D shared  ???
0x65000xxx 0x2F shared USB Function
0x67000310 0x22 shared  ???
0x67010000-0x67010003 - - GCFR register for DMA channels 8, 9
0x67010004-0x67010023 0x34 - DMA ch 8
0x67010024-0x67010043 0x35 - DMA ch 9
0x6B000000-0x6B00003B - - Configuration of shared interrupt requests
0x6B000080-0x6B0000BB - - Status of shared interrupt requests (ints 0x1B, 0x1C, 0x1D, 0x1E, 0x24, 0x28, 0x29, 0x2A, 0x2B, 0x2C, 0x2D, 0x2E, 0x2F, 0x30, 0x32)
0x6B010000-0x6B0101FF ? - HDMI transmitter
0x6B010400-0x6B0104FF ? - HDMI CEC
0x6F0006DA 0x2C shared  ???
0x6F008038 0x2C shared  ???
  • All FR80 registers are big endian.
  • "shared" means that same interrupt number is used. The source of interrupt can be discovered by examining correspondent shared interrupt status registers at 0x6B000080-0x6B0000BB.

Reload 16-bit timer

Appears be same as in MB91605A.

Reload 32-bit timer

Are similar to reload 16-bit timer, but registers are wider:

Offset Description
+4 reserved
+8 TMR
+C reserved TMCSR

Cache controller

Registers appear to be same as in MB91605A. Registers ISIZE, IFUNC, DSIZE, DFUNC are not implemented.

Serial interfaces

Serial ports act same as "Multi-functional serial interface" of MB91605A, but FIFO size is 128 bytes.

Offset Description
+8 ISMK ISBA reserved

External Bus interface

ASR0...ASR5 registers are similar to those from MB91665 datasheet, with following extensions:

  • bit 8 set to 1 mean that there is additional at the end of CS area with the same size. Contents of this area are not defined (sometimes mirror of CS area).
  • bit 3 set to 1 changes scale of bits ASZ0..3 to following:
ASZ3...ASZ0 Description
1000 0x200 bytes
1001 0x400 bytes
1010 0x800 bytes
1011 0x1000 bytes
1100 0x2000 bytes
other values not tested yet

ACR0...ACR5 registers are similar to those from MB91665 datasheet.

AWR0...AWR5 registers are similar to those from MB91665 datasheet with extensions (reserved bits have a meaning now).

DMA controller

DMA channel registers appear be same in MB91605A.

External interrupt controller

It works similar to one in MB91605A, but each external interrupt is mapped to separate interrupt vector.

Resolution converter

Copy/resize plain. Real scaling is done during resize operation.

Offset(hex) Type Description
+0 16-bit Enable register ?
+2 16-bit Command register
+4 16-bit  ?
+6 16-bit  ?
+8 16-bit  ?
+A 16-bit  ?
+C 16-bit Interrupt status register
+10 16-bit Scaling factor 0
+12 16-bit Scaling factor 1
+14 16-bit sourceImageWidth * 100h / targetImageWidth
+16 16-bit sourceImageHeight * 100h / targetImageHeight
+18 16-bit Target image width
+1A 16-bit Target image height
+20 16-bit Source plane width
+22 16-bit Target plane width
+2C 16-bit Source memory address: lower 4-bits
+2E 16-bit Target memory address: lower 4-bits
+30 32-bit Source memory address: lower 4-bits are set to 0
+40 32-bit Target memory address: lower 4-bits are set to 0

JPEG decoder

Offset(hex) Type Description
+0 8-bit  ?
+7 8-bit READONLY: JPEG height from header, higher byte
+8 8-bit READONLY: JPEG height from header, lower byte
+9 8-bit READONLY: JPEG width from header, higher byte
+A 8-bit READONLY: JPEG width from header, lower byte
+F 8-bit Interrupt status register
+400 16-bit  ?
+402 16-bit  ?
+404 16-bit Command register
+40A 16-bit  ?
+410 8-bit Operation error code register
+F00 16-bit  ?
+F02 16-bit Width of Y output plane
+F04 16-bit Output image width
+F06 16-bit Output Image height
+F08 16-bit Width of Cb/Cr output plane
+F0C 16-bit  ?
+F0E 16-bit  ?
+F10 32-bit Target address for Y component
+F14 32-bit Target address for Cb component
+F18 32-bit Target address for Cr component
+F1C 16-bit  ?
+F20 16-bit  ?
+F24 32-bit JPEG file size
+F28 32-bit JPEG file address
+FF4 16-bit  ?
+FF6 16-bit Transfer interrupt status register

Image Transfer Circuit

Copy/fill plain. Can't scale - only able to clip or extend with given color.

Offset(hex) Type Description
+0 16-bit Enable register
+10 16-bit Interrupt mask
+12 16-bit Interrupt status
+100 Channel 0
+100 16-bit Command register
+104 32-bit Source image mask?
+108 32-bit Destination buffer mask?
+102 16-bit Source & destination image mask lengthes ?
+10C 16-bit Source buffer width
+10E 16-bit Destination buffer width
+110 16-bit Destination image width
+112 16-bit Destination image height
+114 32-bit Source buffer address
+118 32-bit Destination buffer address
+11C 16-bit  ???
+140 Channel 1
... ... same as above

SD Host Controller

SD card Host Controller registers fullfill SD Host Controller standard specification. Each access is converted automatically from big endian order of FR80 CPU to little endian SD Host (or back), so no additional conversion is need.

Debayer unit

Converts from bayer image to YCbCr 4:2:2 image.

Offset(hex) Type Description
+1A 16-bit Positive exposure compensation
+3C 16-bit White balance Gr value
+3E 16-bit White balance R value
+40 16-bit White balance B value
+42 16-bit White balance Gb value
+48 16-bit Image width
+4A 16-bit Image height
+FF00 16-bit Command register
+FF02 16-bit Interrupt status register
+FF08 32-bit Source image address
+FF0C 16-bit Source image buffer width
+FF10 32-bit Image Y address
+FF14 32-bit Image Cb address
+FF18 32-bit Image Cr address
+FF1C 16-bit Image Y buffer width
+FF1E 16-bit Image Cb buffer width
+FF20 32-bit Thumbnail Y address
+FF24 32-bit Thumbnail Cb address
+FF28 32-bit Thumbnail Cr address
+FF2C 16-bit Thumbnail Y buffer width
+FF2E 16-bit Thumbnail Cb buffer width

Firmware Tasks

Overview of D5100 firmware B b640101b.bin tasks

ID(hex) Name Description
1 Write files to SD Card Perhaps saving photos made during LiveView
3 Top LCD Manages top side LCD. Only started on D7000.
4 HDMI CEC API Service CEC action requests, process CEC events
5 HDMI CEC Driver Operate CEC device
6 JPEG encoder/decoder #1 API Service requests
7  ? (image device)
8 JPEG encoder/decoder #0 API Service requests
9  ? (image device)
A Image Block Transfer #2 API ? Service requests
B Resolution Converter #0 API Service requests
C Resolution Converter #1 API Service requests
D Resolution Converter #2 API Service requests
E  ? (image device)
F Image Block Transfer #0 API Service requests
10 Image Block Transfer #1 API Service requests
13 Audio API Service audio requestes
14 Audio DMA API Service requests for Audio transfer
15 Play Beep Playing subject in-focus sound
16 Init First task started at startup after mITRON OS kernel was initialized
17 IMX071 API Communication with CMOS photo sensor IMX071
18  ? Main preprocessing of video images ?
19 Interconect FR <-> TX communication service
1C USB Main Connection detection, start/stop conditions.
1D USB Device PTP or Mass Storage USB Device Service
20 Movie Frame Handler Resize video frames and pass to codec
24 Audio Video Manager Switch audio/video devices, send frames to TFT, HDMI, etc
25 Buttons Buttons check and actions
26 Detect audio/video/sdcard plug
27 USB Async Event Generate USB Interrupt Request
28 PictBridge DPS Service
29 WiFi (WLAN) Service Only started on D7000
2A SD Card API Detect card, service requests as format, etc
2B GUI TFT Graphical User Interface: menus, LiveView, etc
2C TFT double buffer Flip tftOutputBufferAddr every 0.5s
2F Copy set2E, set2F, set30
32 EyeFi EyeFi manager
33 Movie Encoder API Video encoder service
34 Movie Decoder API Video decoder service
35 Movie Enterprise Video codec control task
36 Movie Display Movie playback displaying task
37 Movie Controller Control movie recording/playing process
38 Movie MBox API Video encoder/decoder mailbox service
39 Movie Record API Service movie recording requests
3A Movie Play API Service movie playback requests
3B Movie Audio API Movie Audio feed manager

Reference documentation from Fujitsu

The "A" microcontroller

General information

The "A" microcontroller is aimed at I/O tasks. For our target cameras (D5100 etc), it's a Toshiba TMP19A44FEXBG.

Address map

Virtual addresses are used (as it is seen in firmware).

Address range Size Description
0xBFC00000-0xBFCBFFFF 768KB Internal flash memory. Contains the firmware itself.
0xFF000000-0xFF007FFF 32KB Contains I/O registers for peripherials. See hardware documentation below.
0xFF200000-0xFF3FFFFF 2MB Reserved for debuging purposes (PrAcc access mode).
0xFFFF0000-0xFFFF3FFF 16KB Internal backup RAM (contents not lost on RESET).
0xFFFF4000-0xFFFFFFFF 48KB Internal RAM (contents lost on RESET).

Firmware Tasks

Overview of D5100 firmware A a640m010100.bin tasks

ID(hex) Name Description
1 Init First task started at startup after mITRON OS kernel was initialized. Started on each change of Power State like up/down/low-power.
2 GPS Process incoming GPS sentence, debug.
4 Manager Control Viewfinder, flash chip (NHHS), power-save logic
5 EEPROM API Service EEPROM requests: read, write.
6 Lens API Service lens action requests.
8 Interconnect Service FR <-> TX communication
A  ? Multiuse
D Power API Service requests: measuring battery/power supply voltage level, etc

Reference documentation from Toshiba