The architecture of the TuMag camera FPGA blocks diagram is shown in
Figure 1. It consists of two main blocks: the
sensor driver module and the
TuMag embedded control system module. The
sensor driver module is responsible for direct communication with the CMOS detector. It defines an
SPI interface to access the internal registers of the sensor, a finite state machine circuit that generates 19 timing control signals to read the sensor data synchronously, and a circuit to read and convert 8 serial LVDS 12-bit data channels to 12 parallel words. In addition, the
sensor driver has implemented a hardware circuit to calibrate the sensor, channel order, readout electronic control, etc. The
TuMag Embedded Control System (TuMag-ECS) module is a core-based MicroBlaze soft processor that manages the 3125 Gbps CoaXPress communication interface [
9] between the host (or frame grabber) and the sensor. It also serves as a communication interface between the heater and the temperature controller sensor, monitors the supply 24 V voltage of the TuMag camera, and switches the sensor on/off. Additionally, the TuMag-ECS, through the AXI-Lite and AXI4 video stream buses, sets and manages the communications of the
sensor driver to configure the sensor and/or read the data. The TuMag camera firmware was implemented in an Artix 7 FPGA using VHDL and Verilog hardware description language. The Artix 7 FPGA device provides the highest performance-per-watt fabric, transceiver line rates, and DSP processing in a cost-optimized FPGA [
10].
In addition, TuMag-ECS is in charge of bit, word, and serial channel calibration and of the modification of several parameters, such as the region of interest (ROI) size, exposure time, hardware or software trigger, single or continuous acquisition modes, sensor gain, and adjustment of the black level offset.
The sensor GSENSE400BSI is a scientific [
6,
7,
8], back-illuminated CMOS image sensor with a wide dynamic range of pixels produced by Gpixel INC. The sensor has the advantages of high sensitivity, low noise, and high integration, including a 12-bit ADC, a temperature sensor, a PLL, and an SPI controller on-chip. The ADC is used to convert analog signals into 12-bit digital signals, and the PLL is used for on-chip clock management. The SPI controller is used for on-chip register configuration, which can change the on-chip PGA gain, data readout direction, PLL crossover multiplier, and operating mode. In addition, the CMOS sensor has two modes of operation: standard (STD) mode and high dynamic range (HDR) mode.
Table 1 lists the key specifications of GSENSE400BSI.
2.1. TuMag-ECS Architecture
The TuMag-ECS was implemented with the Vivado tool (
Figure 2); in the
Supplementary Material, we added a PDF file with the block diagram of all TuMag-ECS circuits. The main task of this system is to control both the GSENSE400BSI
sensor driver (right side) and the CoaXPress communication interface with the host or frame grabber (left side). In addition, it also controls/monitors other modules (IP) necessary for the instrument, such as the
heater module (temperature controller), the
SPI module to read the internal sensor registers, the
train module to put the sensor in training mode, the
AXI_to Camera register necessary to configure the sensor, the
XADC module to monitor internal FPGA parameters, and, finally, the
Easii_cxp_dev controller of the CoaXPress communication interface.
This embedded system is based on an architecture defined by an AXI4-Lite bus, which, by means of the software drivers of each module, allows carrying out the necessary reading and writing operations for the following tasks: monitoring the heater status, monitoring the supply voltage of the system, configuring the operating modes of the sensor, and managing all other functionalities defined in the firmware of TuMag ECS. These drivers are implemented through simple software functions executed on a MicroBlaze soft-processor IP core (2017.1) provided by Xilinx. In this processor, the software control application controls/decodes the communications between the sensor and the host and vice versa. Additionally, it manages all IPs belonging to the architecture of the embedded control system. These IP cores are the heater control, spi, AXI_to_camerareg, sensor train, PWR, Cxp device IP, and Cxp Transceiver controller. The latter is connected to the FPGA GTP transceiver interface.
2.1.1. CoaXPress Communication Interface
The data transfer from the CMOS sensor to the instrument host is performed via a CoaXPress interface. This interface connects devices (typically cameras) to a host (frame grabber) and allows, over the same cable (single coaxial cable), data transfers plus commands and power. The CoaXPress standard is a fast, point-to-point asymmetric speed (20.833 Mbit/s to uplink and up to 6 Gbit/s to downlink) 8B/10B encoded packet serial communication [
12,
13]. On each connection, the link protocol defines a set of logical channels, which are used to send trigger signals, general-purpose inputs and outputs, control data, and high-speed streaming video transmission. Each device connected to the interface is controlled through the control channel by the host.
Figure 3 shows the CoaXPress interface modules (
Cxp IP device and
CxP Transceiver Controller). These connect the image sensor data from the CMOS sensor (
Figure 2, right side) with the host or frame grabber (
Figure 2, left side).
Figure 3a shows the interface ports of the
Cxp device module. It has a control AXI4-Lite interface, which is used to monitor and control all user memory space, and an AXI4 slave video stream interface dedicated to image data transfer from the TuMag
sensor driver to the host.
Table 2 shows an interface port description. The uplink and downlink data interface is connected to CoaXPress through the Artix 7 FPGA GTP transceiver and
Cxp Transceiver Controller module, see
Figure 3b.
Table 3 shows interface port descriptions. The output of the GTP transceiver is connected to physical media through a Microchip chip [
14].
Additionally, the CoaXPress interface supports the Generic Interface for Cameras or GeniCam standard [
15]. It permits decoupling industrial camera interface technology such as CMOS sensor GSENSE400BSI from the user application programming interface (API). This standard support is conducted through an XML file stored in the device. This file is shared with the host at the time of connection and thus provides a description (namespace given by the standard) of the GeniCam API-compliant device (GenAPI). The boot records, stored on the
Cxp device, provide the frame grabber with a way to access the XML file.
The TuMag-ECS CoaXPress communication interface has been tested using a commercial frame grabber installed on a PC (Euresys Coaxlink Quad frame grabber). Through it and for debugging purposes only, we have checked the operation of all the camera functionalities.
2.1.2. Heater Module
This module (
Figure 4a) is necessary for the thermal control of the CMOS Sensor. Basically, a Peltier/heater device is included in the sensor PCB. The thermal control is performed by means of 4 commands managed by the host. These commands are sent from the frame grabber or host and the software of the TuMag embedded control system decodes them into local actions like setting the temperature setpoint value, enabling and disabling the heater, or reading the heater status. The
heater control IP core configures, via SPI, the Texas Instruments DAC8311IDCKT [
14] (
Table 4). The SPI clock is limited to a maximum frequency of 1 MHz due to path restrictions. The
heater core has a timer configured to periodically (every second) refresh the DAC to prevent it from losing its configuration and desired temperature and set point value.
2.1.3. SPI Module
For the GSENSE400 BSI CMOS detector, after the power-up or reset sequence, the default values for all on-chip registers are ‘0’. These registers should be re-programmed. The sensor driver module (
Figure 1) has an integrated FSM (finite state machine) to perform read and write (re-programming) operations on the 256 bits of the internal register. Both operations are performed at a frequency of 25 MHz coming from the detector. These read-and-write sequences cannot be interrupted by each other, i.e., all 256 bits have to be read or written without interruption.
In the write operation, the FSM takes the data from the RB2 register and reprograms the internal register of the sensor. In the read operation, FSM reads the data from the sensor (organized as 32 8-bit registers) and stores it in the SPI module. The architecture of this IP is based on a dual real port BlockRam. It is wrapped with an AXI4-Lite interface on the MicroBlaze side and an 8 data bit on the
SPI sensor side (
Table 5). In this way, the memory decouples the clock domain of the sensor driver from the clock domain of the AXI4-Lite interface.
Figure 4b shows the IP interface.
2.1.4. Train Module
The CMOS sensor has a train operation mode that permits alignment/calibration of the incoming 12-bit 8-channel serial data from the sensor. Training is run at the start of the firmware embedded software or upon request of the host. This training task is performed in the
sensor driver block (
Figure 1) and monitored by the
train module (
Table 6). When bit, word, and channels alignment is achieved, a status report is generated and sent to the instrument host. The training is repeated until all data channels are calibrated.
Figure 5a depicts the
train module interface.
2.1.5. XADC Module
This IP (inside of FPGA device) was used to monitor the internal FPGA on-chip register values such as on-chip temperature reg, on-chip VCCINT data reg, on-chip VCCAUX data reg, ADC out of VP/VN, on-chip VREFP data reg, on-chip VREFN data reg, and VBRAM data (
Table 7).
Figure 5b shows the IP block port interface. These measurements need to be known to be able to properly correct camera frames from dark current or gain fluctuations.
Figure 5b depicts the XADC module interface.
2.1.6. AXI_to_Camera Register Module
The embedded software application configures and controls the operation of the CMOS through the
AXI_to_Camera register module (
Figure 6). To do this, the module has three register memory banks, RB1, RB2, and RB3, for storing the configuration parameters of the sensor driver. The RB1 register bank is used to control the data acquisition process, set the sensor in calibration, acquisition, or test mode, store the calibration words, and store the words for testing the sensor in test mode. The RB2 register bank stores the 256 bits sensor register for configuration through the FSM circuit integrated in the sensor driver module. RB3 is a register bank used by the image decoder module (in the sensor driver block) to generate the timing control signal for image acquisition.
Table 8 shows the interface ports description of the camera register banks module.
2.2. TuMag Embedded Software Application
The TuMag camera has a set of hardware registers through which it is possible configure the acquisition process, as well as the conditions under which it is conducted. In addition, the camera has another set of registers that allow the monitoring/controlling of other system parameters, such as the FPGA input voltage or the internal sensor temperature.
The embedded application software was developed using Xilinx SDK IDE in the C programming language and under the GeniCam standard. This application, which runs continuously on the MicroBlaze soft processor, has the functionalities of initializing local FPGA registers into driver module and sensor configurations FPGA register banks (RB1, RB2, and RB3), establishing the communications system link and speed between the frame grabber (host) and the device for data acquisition (sensor), updating (to shared) the XML file (GeniCam camera standard specification) on the host, setting the interrupt map in the embedded system memory, and decoding the commands coming from the host.
Figure 7 depicts a simplified flow diagram of the main code. We can differentiate two stages: the initialization stage (
Figure 7 to the left) and the CoaXPress link-command decoding stage (
Figure 7 to the right).
In the initialization stage, the application sets the CMOS sensor to the reset state, configures the CoaXPress interrupts map on MicroBlaze memory, builds and initializes the camera register banks (RB1, RB2, and RB3) on memory with default values, updates the RB1, RB2, and RB3 FPGA register into sensor driver modules, and configures the CMOS sensor with default values of the RB2 register.
In the CoaXPress link-command decoding stage, the application goes into an infinite listening loop where the device–host is linked through a discovering process, where the communication speed is set to 3125 Gbps, and where the GeniCam XML file format of the TuMag camera is exchanged with the host. This XML file, defined under the rules of the GeniCam standard, defines the configuration parameters, i.e., features of and commands of the TuMag camera, and permits the decoupling of sensor technology from the host application. The XML file, compressed in zip format, used in this design is organized in three categories: device control, image format control, and acquisition control. Within each of these categories there are different entries or registers. Each register, defined according to the GeniCam standard, has a name, an address, an access mode (Read only or Read/Write), a data type (command, integer, float, string, enumerated), and valid or allowed values.
Table 9 shows some input values in the three categories.
The command decoding algorithm works like a “look at table”. In the MicroBlaze memory, all GeniCam entries used are defined with their name, address, and value. The software listens for an interruption coming from the host. When this happens, four parameters are read from the CoaXPress device, the device identifier, the command (cmd), the address (addr), and the size of the message (size). From the value of the cmd, the algorithm decides whether it is a write or read operation.
In the first case (writing cmd = 0 × 01, hexadecimal format), the data are read from the CoaXPress device y with the addr variable, and the “look at table” is searched to update the parameter (i.e., image) with the data value. The write-host commands permit updating the local FPGA registers RB1, RB2, and RB3, setting the temperature value of the heater circuit, configuring the internal CMOS registers, running the sensor training process, setting the camera in acquisition (continuous, single frame or CoaXPress) or test mode, and setting the sensor power ON/OFF.
In the second case or read command, cmd equals 0 × 00, the variable address is used to read the parameter and store variable data to be sent to the host (i.e., addr = 0 × 3028 data < −300 ms). Just like the write operation, the host can access the FPGA parameter values.
In both cases, write and read operations, the algorithm command decoding handles the errors through an ack variable in the following way. By default, the value of the variable is an invalid address or FALSE. If the decoded address is valid, the ack variable is updated as a valid address or TRUE. At the end of the address decoding process, the ack variable is sent with the appropriate value.
In the camera observation mode or CoaXPress mode (24 frames per second), image acquisition is controlled directly by the host. The principle of operation of the acquisition process is as follows: the host sends the trigger command and the CoaXPress device (
Figure 2) generates the
HW trigger signal and sends it to the sensor driver.
This signal is used to start or enable an FSM that controls the sensor readout cycle. This cycle consists of three phases: resetting rows, waiting for the configured exposure time, and eventually reading out in row order [
14].
Figure 8 depicts a VHDL hardware simulation of the acquisition process in the sensor driver of
Figure 2, showing only 32 rows for the compressed view. After the HW trigger signal is received, the sensor driver module controls the acquisition process by generating the image request signal (
frame_req set to high red signal in
Figure 8) and generating the time signals to synchronize the data readout (blue signals in
Figure 8). The
sync_x signal sets the data reading rate on the GPIXEL sensor device and, when the
fval signal is set to 1, controls the time period where the data are available to be transmitted to the host. After the last line, the
lval signal is transmitted, the
grab_done signal (red in
Figure 8) is set to high, and the sensor driver control goes into an idle state, waiting for a new HW trigger or command from the host.
2.3. Test Bench Software Application
Each flight and spare camera are later subjected to the following test campaign to ensure its functionality:
Optical characterization under ambient conditions (∼25 °C and 1 bar).
Optical and electrical characterization under near-vacuum conditions and three temperatures (hot, nominal, and cold). This provides confidence that the camera behaves correctly if there is no degradation with respect to the first step. No early component failures (“infant mortality”) have arisen from that first stress.
A bake-out for the elimination of all particles that could degrade the optics of the instrument. The camera is not yet supplied but kept at 60 °C for 72 h and 5 mbar of pressure.
A thermo-vacuum test (three temperatures: hot, nominal, and cold) to verify that the camera temperatures meet the thermal requirements (image sensor, FPGA, and cold finger temperatures are read by the camera itself; the temperature sensors placed on the camera cover are read by the test setup).
Finally, a second optical characterization gives us confidence in its behavior, before delivering the camera for assembly into the instrument.
The TuMag cameras (
Figure 9) are a critical subsystem of the instrument. To avoid risks in the transport of the complete instrument between the different laboratories where the testing campaigns were carried out, we designed and implemented a test-bench software application (TBSA) to handle the TuMag camera from a local computer. This allowed us to simplify the number of devices to be transported and reduced the assembly time of experimental setups. Due to the fact that each laboratory of the research consortium has this tool at its disposal, we only moved the camera and its associated electronics between the different laboratories to be characterized.
This TBSA, formed by a set of methods developed in Python and Java scripts, handles the Coaxlink Quad card, a trade frame grabber provided by Euresys Company installed on a personal computer. This card permits the streaming of images and configures, controls, triggers, and powers multiple high-speed cameras via CoaXPress 1.1 interface technology and GenICam programming interface (
Figure 10).
Euresys Company provides its own GenAPI script language (like JavaScript) for programming modules and scripts to handle the features of frame grabbers and cameras based on GenICam. To adapt this generic software architecture to the TuMag camera, we implemented the
tumagLib.js and the
featuresHandler.js scripts. The first file defines a set of functions implemented to read, write, and execute the TuMag features. The second file contains the program used to handle the instrument.
Figure 11 shows a block diagram representing the hardware and software architecture of TBSA used to handle the TuMag camera.
When the featuresHandler.js is launched, the frame grabber and the TuMag camera are automatically detected first, and then the script is executed. To write, read, and execute different TuMag features, this script must be modified to add the corresponding functions defined in the tumagLib.js module. For this, the user can use any text editor. In particular, the featuresHandler.js script is executed using the GenTL Console tool, which includes a GenTL interpreter specific to the Euresys GenAPI script language.
In addition, to improve the interaction and interface between the user and the hardware, we have developed a wrapper library developed in Python. Specifically, the wrapper consists of two files:
tumagDef.py and
tumagLib.py (
Figure 11). The first one contains a list of the function names that are defined in the
tumagLib.js module. It also includes the definition of paths to the necessary program files and required opcodes to generate system instructions automatically. The second file contains a set of functions to access and configure each of the available TuMag features. These Python functions automatically modify the script contained in
featuresHandler.js to add the appropriate code and directly execute the corresponding system instruction to launch the Euresys GenTL interpreter.
This Python library is formed by the get, set, and execute functions, as well as initlib.
Table 10 shows the Set/Get actions of the library grouped by functionalities. We have developed a user guide (the document is available on request from the authors) where the installation of the library, CoaXPress card and drive installation, and examples of the utilization can be found.
The results are displayed in the terminal console. As an example of use,
Figure 12 shows a block diagram of steps to measure Dark Image Characterization. The measurement process, defined in the Python script, is a succession of commands sent by the host to be decoded by the TuMag camera. It ends when all the gains and integration times are executed. Acquired images are sent to the host as they are captured.