NikonEmulator

(formerly FrEmulator)

What is it ?
Trying to understand how the firmware(s) inside Nikon DSLR work, we thought it could be useful to have a software that would take a firmware as input, then imitate what a microcontroller would do with such instructions and data. That's what is generally known as "emulating" a device.

The NikonEmulator started as a simple interpreter of the Fujitsu FR ("B" microcontroller) instruction set (hence the former FrEmulator name), simulating only an FR CPU and its memory. Then, as understanding progressed, we added several "components", as well as tracing and debug tools.

Lastly, we thought it would be good to also study the Toshiba chip ("A" microcontroller) and we started over to emulate that chip too. Consequently, version 2 onwards of the emulator really contains two completely separate emulators and toolsets, and was renamed NikonEmulator.

This is a work in progress, obviously.

Getting Started

 * 1) Requirement: To run the latest versions of the NikonEmulator, you'll need a Java 7 environment or later (aka Java 1.7+). If you don't have one, please Download and install a Java SE from Oracle's website. The NikonEmulator is known to work under Windows and Linux, but any platform with a Java 7 environment should be OK.
 * 2) Download the latest version of NikonEmulator-x.xx.zip from this Google Drive link and unzip it to any folder.
 * 3) Download an official firmware from the Nikon website. It is advised to use the D5100 v1.01 firmware (F-D5100-V101W.exe) that serves as a reference for research currently.
 * 4) Open that firmware file with an unzipping tool (such as 7-zip for Windows, and extract the D5100_0101.bin file it contains. That's the encoded firmware.
 * 5) Start the emulator by double clicking the startEmulator.bat file (or startEmulator.sh under Linux or Mac OSX)
 * 6) Decode the encoded firmware by using "File > Decode firmware" menu, selecting D5100_0101.bin as the source file, and (for simplicity) NikonEmulator's own directory as target. You should get two files named "a640m010100.bin" and "b640101b.bin". Those are the decoded A and B firmwares.

You are now ready to start hacking.

Uninstalling
The NikonEmulator has no installer and all files are in the unzipped folder, except for a config file that gets created upon first run. This file is called .NikonEmulator and is stored at the root of the user's home directory, e.g. C:\Users\ \.NikonEmulator on recent Windows or ~/.NikonEmulator on Linux.

Consequently, full uninstallation is achieved by deleting the unzipped folder and above config file.

User interface
The DSLRs we are interested in share responsibilities between two distinct chips (known as A and B. See Understanding Firmware for more information). Consequently, the NikonEmulator doesn't emulate one chip, but two. This is the reason for the dual-pane interface.
 * On the left hand side, with a light blue background, sits the Expeed emulator. The Expeed, or 'B' chip, is based on a Fujitsu FR80 family microcontroller, abbreviated FR80 throughout the interface.
 * On the right hand side, with a light green background, sits the I/O chip emulator. The I/O chip, or 'A' chip, is a Toshiba microcontroller of the TX19A family, abbreviated TX19 throughout the interface.

As you can see, both panes look very similar, with the exception of a few features only applicable to one or the other chip.

You can freely move the vertical separator between the panes, or even completely hide one of the panes by clicking on the small arrows on the top of the separator.

Features
The emulator features can be classified in the following groups (roughly corresponding to the application menu):
 * File management. This is where you can decode firmware from the cyphered versions provided on Nikon's Website. There is also a basic way to save and reload emulator state, but please be aware that it is limited to memory and CPU, so it can only be used for small tests, not for a complete session.
 * Code emulation itself (Run). They correspond to the "Transport" button group (play/pause/etc.). This allows you to execute firmware instructions in different ways, either one-by-one or in sequence, with an optional variable delay between instructions. If the emulator executes instructions in sequence, you can pause it at any time with the corresponding button, or by setting up breakpoints (more on this later). The "STOP" button corresponds to a reset (it is advised then to close all windows that could still reflect obsolete states).
 * Component inspection and interaction, grouped under the "Components" menu. Emulation involves, at the very least, a CPU and memory. Each instruction executed by the [CPU has an effect on its internal state (registers) or memory (either actual memory, or memory space mapped components). But other components are also required for a microcontroller to run: Interrupt controller, Timers, etc. Each of these components corresponds to a given Window in the emulator, allowing you to inspect the state of that component, or sometimes even force a given state.
 * Tracing. This is about keeping track of what happens. For example, which instructions were executed (real-time disassembly log), what is the current function call nesting (call stack) or what memory areas are being accessed.
 * Source code. This is how you can perform a full disassembly and basic analysis of the firmware code, navigate the resulting assembly code and display a call graph of the functions found
 * Tools. This is where all the rest fits, among other a feature to dump part of the emulated memory to file, or reload it, as well as options used for disassembly and general User Interface configuration.

Sample uses
Note: The following use cases assume that you have already performed the "Getting started" steps above.

Getting a firmware disassembly
These are the steps to disassemble one of the firmwares (replace [chip] by FR80 or TX19 below or use the buttons on the corresponding side):
 * 1) Open the binary firmware using "File > Load [chip] firmware image" or the "Eject" button
 * 2) Select your options using "Tools > [chip] options" or the "Gear" button
 * 3) Open the code disassembly window using "Source > Analyse / Disassemble [chip] code" or the "Screwdriver" button
 * 4) Select an "options" file, which contains code structure definition and identified symbols. Sample files are provided for the D5100 firmware in the "conf" folder of the NikonEmulator distribution. They use the same base name as the firmware itself, followed by dfr.txt or dtx.txt depending on the chip you work on
 * 5) Select a target file to write disassembly to (optional)
 * 6) Select the options you want (recommended are "structure", "parameters", plus "int40" for the FR80 chip)
 * 7) Press OK. A log should open, containing many warnings but no errors. This is normal.
 * 8) You can now navigate the code using "Source > [chip] source code" or the "Source code" button
 * 9) If you chose a target file, you can also open that file in a text editor. Warning, the file can be large. Notepad++ or equivalent is recommended

Eye candy
Note: only works with version 2.51 or later.

To achieve this, the following steps must be performed: After the emulated time value (shown in the status bar) reaches 1800ms, and you will get the screen on the right.
 * 1) Open the A binary firmware using "File > Load TX19 firmware image" and choose "a640m010100.bin"
 * 2) Open the B binary firmware using "File > Load FR80 firmware image" and choose "b640101b.bin"
 * 3) Open the camera front panel window using "Components > Front Panel (TX only)" and turn the camera on by clicking on the power switch (it will turn red)
 * 4) Open the I/O ports window using "Components > TX19 I/O ports", select the "Values" tab, then set P51 (row "IOPort 5" and column "bit# 1") to the value "VCC" (HI)
 * 5) Open the camera screen window using "Components > Screen emulator (FR only)"
 * 6) Start the emulator using "Run > Start (or resume) FR80 emulator" or the "Play" button.

Note that this error message is expected (because an eeprom contents is blank), but getting to this point proves that both processors are emulated correctly, and that they communicate with each other.

Tracking memory access
The emulator has several ways to log or spy memory accesses while code is running:
 * 1) the first one is the "Memory activity viewer" (that's the black square window button). It should be open before letting the code run. Each cell represents a 64k-page and its color reflects the access type operated on that page (red for write, green for data read, blue for code read, making up a RGB value), and each primary color is incremented when one operation happens. In other words, if a page is read 200 times for code and read 50 times for data, it will look green with a slight cyan tint (RGB=0,200,50). Currently, when 255 is reached, the value doesn't change anymore (in an alternate implementation, values went back to zero, causing repeated access to make the areas flash). If one clicks on a given pixel in that window (with the help of the pop-up status for precise positioning), a new sub-window opens, representing accesses inside that 64k-page. In other words, it "zooms" and the new square has now one cell per address, so you can see what happens at the byte level.
 * 2) the second one is the hex editor window. Indeed, if the "Memory activity viewer" window above is left open, then when the hex editor window is opened, the bytes it shows will be colored according to same color scheme (except there is no increment: a byte read at least once is green, etc). This view can be seen as a "mega-zoom", as each cell in the previous view is now a hex byte.
 * 3) the third one is the "Custom logger" window, where one can define several start/end address, and every read/write access inside those address ranges will be logged to the window.
 * 4) the fourth one is the "Watch trigger" which "traps" single write operations. To capture changes at a given address, first open the Hex editor window, and switch to the "Watches" tab. Click "Add" and double click on the "Address" column to indicate the address you want to watch, and on the "Name" column to give it a meaningful name. Then click "convert to trigger" and a new trigger will be created in the "Breakpoints" window, so that the emulator will stop when the value at that address changes.

General
Of course, the ultimate goal of the emulator is to perform a full boot sequence and, hopefully, one day, behave as a full Nikon camera would. The boot sequence is probably one of the hardest process to achieve, because all the software (OS) and hardware has to be initialized and checked. Here are the first steps to be performed for each CPU:
 * 1) Open the binary firmware using "File > Load [chip] firmware image" or the "Eject" button

TX19-specific init
On the TX19 side, we have discovered that a given pin (port5 bit1) has to be at the "high" level for the boot process to succeed. So: The TX19 processor also talks to a serial Eeprom whose contents has to be loaded beforehand. So: To avoid having to perform the above steps each time you start a session, do the following:
 * Open the I/O ports window using "Components > TX19 I/O ports" or the corresponding button.
 * In the "Values" tab, set P51 (row "IOPort 5" and column "bit# 1") to the value "VCC" (HI). You can then close the window. Settings are persisted across sessions.
 * Open the EEprom window using "Component > TX19 Serial Devices (TX only)".
 * In the "Eeprom" tab, select the "Contents" sub-tab.
 * Click "Load..." and select the D5100-eeprom.bin file included in the emulator folder. You can then close the window.
 * Open the TX19 Options window using "Tools > TX19 Options".
 * In the "Eeprom Options" tab, select "Last Loaded". You can then close the window.

Starting emulation
To make sure we achieve reproductible results, we have added a way to start both processors at the same time and keep them in sync no matter if one hits a breakpoint or performs a time-consuming task from the emulator's point of view. So: To observe the exchanges, please do the following: You are now ready to start:
 * Please make sure the "Keep emulators in sync" checkbox at the top right is checked
 * Click on the "Components > TX19 serial interfaces" menu and select the "TX19 HSerial #0" tab in the opened window.
 * Start the emulator using "Run > Start (or resume) [chip] emulator " or the "Play" button on any side of the screen

Observations
Both OSes are now running as can be observed by opening the µITRON operating system objects window using "Trace > µITRON [chip] Objects" (or the "3 cubes" button) and pausing at different times.

After some time (around 540 emulated milliseconds, see status bar), one can observe a synchronous serial exchange happening between the chips (see Serial Protocol). The TX19 (master) sends 2 zero bytes (00 00) to the FR80, which posts the 01 03 header in return. Then the TX19 sends a 0xFF (which is the checksum of its first two bytes "00 00"), plus 3 more zeroes (00 00 00) In return, the FR80 chip replies synchronously to those bytes with its message payload and checksum (31 30 31 69) The TX19 chips analyses this message and checks it is correct, so it sends back the received function ID (first byte) as an acknowledge (01) and FR80 replies with the TX19's message function ID (00). In case of CRC error, one could have replied with 0xFF.

Then, after more time, more exchanges go on between chips

A note on breakpoints and triggers
For people used to development environments, the concept of Breakpoint will sound familiar. Basically, by hitting the "Debug" button instead of the "Play" button, you instruct the emulator to pause the program whenever it is about to execute some instructions you "marked" beforehand. Those "marks" are breakpoints. Such breakpoints can be set by navigating the source code (after disassembly, see first use case above), then right-clicking on an instruction and selecting "Toggle breakpoint".

Alternatively, the breakpoints can also be defined or edited in a dedicated Window accessed using "Run > Setup [chip] breakpoints" (or "Breakpoint" button).

You will notice that when you Add or Edit a breakpoint, the window allows you to specify much more than just a PC address. This is because NikonEmulator breakpoints can do much more than pause on a given instruction:
 * they can break when any CPU register matches a given value, and can combine several such register conditions
 * for registers presented in binary form (such as the FR's CCR register), you can specify which bits you want to match by using a combination of "1", "0" and "?" (don't care) characters
 * you can also break when a given memory position (optionally masked) matches (or doesn't match) a given value, or more generally when it changes.

Emulator is optimized in favor of PC-based breakpoints. In other words: emulator performance is much better in "debug" mode using only breakpoints on specific "execution address" (or PC).

Finally, although the primary goal of a breakpoint is to pause code, the concept was extended so that when the conditions match, the breakpoint (let's call it a trigger) can do any combination of: That last option provides a very effective way to skip code that would fail due to missing or badly emulated components, for example.
 * pause code (of course)
 * log the condition match in the log tab
 * trigger a configurable interrupt
 * perform a jump to any part of the code
 * change value of any CPU register

EEPROM
Configured in the TX19 options there are following possibilities:
 * blank EEPROM (each time you start the emulator, ot is filled with zeroes)
 * persistent across session (each time you start the emulator, it reloads the contents that was last saved to the preference file)
 * last loaded (each time you start the emulator, the bin file is reloaded from disk)

Until now, we mostly used that last option, to have reproductible results, but if you select "persistent", then you will always restart with what was last saved to the eeprom in the previous session.

If you want to export EEPROM contents, you can always open the "Serial devices" window on the Eeprom contents tab, then click "Save". That will save a raw binary copy, so you can for example compare it with the original.

Inside the emulator
If you want more information about how this Emulator works, please see the NikonEmulator Internals page.