Abstract
The use of deep learning (DL) algorithms has improved the performance of vision-based space applications in recent years. However, generating large amounts of annotated data for training these DL algorithms has proven challenging. While synthetically generated images can be used, the DL models trained on synthetic data are often susceptible to performance degradation when tested in real-world environments. In this context, the Interdisciplinary Center of Security, Reliability and Trust (SnT) at the University of Luxembourg has developed the “SnT Zero-G Lab,” for training and validating vision-based space algorithms in conditions emulating real-world space environments. An important aspect of the SnT Zero-G Lab development was the equipment selection. From the lessons learned during the lab development, this article presents a systematic approach combining market survey and experimental analyses for equipment selection. In particular, the article focuses on the image acquisition equipment in a space lab: background materials, cameras, and illumination lamps. The results from the experiment analyses show that the market survey complimented by experimental analyses is required for effective equipment selection in a space lab development project.
1. Introduction
In the last few years, deep learning (DL) techniques have been proven successful in vision-based space applications such as satellite pose estimation [1, 2] and spacecraft navigation [3]. However, DL models require a vast amount of annotated data to learn data patterns and achieve a high performance. Nevertheless, due to the difficulties of obtaining large real-space datasets with correct labels, robust space-related DL solutions are currently missing. For that reason, previous research has mostly relied on synthetic data for training DL models [2, 4, 5]. However, while synthetic images are easy to generate and annotate for training DL-based solutions, they are prone to performance degradation when the model is tested in a real-world environment, as DL solutions tend to overfit the features from the synthetic domain [6, 7]. This phenomenon is known as the domain gap problem [8, 9].
To address the domain gap problem, several research institutions around the world, such as the European Proximity Operations Simulator (EPOS) [10] and Autonomous Spacecraft Testing of Robotic Operations in Space (ASTROS) [11], are developing laboratory facilities for mimicking space conditions with the objective of obtaining more reliable datasets. Research suggests that real-world space-like image datasets (Spacecraft PosE Estimation Dataset+ (SPEED+)) [12, 13], SPAcecraft Recognition everaging Knowledge of space environment 2022 (SPARK-2022) [14]) collected in these facilities can be used to train and evaluate the robustness of vision-based space algorithms, mitigate the domain gap, and provide higher confidence on the performance of the DL models when deployed in space [12]. However, the construction of such facility entails a plethora of uncertainties as it is not a standardized nor well-documented process. Consequently, research centers undertaking this endeavour face many challenges, including a lack of support in the form of guides, manuals, or templates [10, 15, 16]. As any other development project, the major drawback from these uncertainties is the increased probability of cost overruns, project delays, and even project failure [17].
In 2019, the Interdisciplinary Center of Security, Reliability and Trust (SnT) at University of Luxembourg undertook the project of developing the “SnT Zero-G Lab,” a facility for mimicking space environment and simulating rendezvous-related processes [18–21]. During the development, SnT Zero-G Lab had faced cost and schedule challenges related to the lack of literature documenting the development of such facilities. This article belongs to a series of articles that SnT is producing with the objective of bringing forward the lessons learned during the development of the SnT Zero-G Lab and supporting research institutions around the world in developing their own space facilities.
An important aspect of building a space facility like the SnT Zero-G Lab is the equipment selection. The selection of the right equipment to emulate a space-like environment and capture images of acceptable quality level in such conditions is crucial. Hence, it is important to have details on the available options in the market, selection metrics to be used, and experimental analysis methods for equipment comparison that would support purchase decisions. In this context, the goal of this article is to provide a systematic approach to support equipment selection and decision-making when developing a space lab for vision-based applications. In particular, the article focuses on the equipment required in the image acquisition process: the laboratory background materials, cameras, and illumination lamps.
The contributions are summarised below: (i)A detailed survey of market-available choices for image acquisition equipment. Background materials, cameras, and illumination lamps were surveyed and compared based on selection metrics. The selection metrics were chosen based on their relevance to the image-capturing process (for example, focal length and shutter speed) as well as the lab development project objectives (for example, cost and size)(ii)Experimental analyses for equipment comparison for background materials and cameras(iii)Together, the survey and experimental analyses provide a systematic approach for equipment selection in the development of similar space labs. The presented framework can be extended to include more products when available in future and make a selection based on different project objectives such as budget constraints and intended applications
The remainder of the paper is organised as follows. First, a literature review of existing space facilities with a focus on image acquisition components is summarized in Section 2. Then, a market survey of laboratory background materials, cameras, and illumination equipment is presented in Section 3. The survey is then complemented with different experiments to analyse the suitability and performance of commercially available equipment. In Section 4, the laboratory setup is described, and in Sections 5 and 6, experimental analyses of laboratory backgrounds and cameras are presented. Section 7 discusses the results, and Section 8 concludes the paper.
2. Related Work
Several facilities providing testbeds for training and validating vision-based space applications exist at different research institutions around the world [22, 23]. In this section, a review of these facilities with a focus on image acquisition components (background materials, cameras, and illumination) is presented.
2.1. TRON, USA
The Robotic Testbed for Rendezvous and Optical Navigation (TRON) facility at Stanford’s Space Rendezvous Laboratory (SLAB) is the first of its kind developed for testing machine learning-based space-borne optical navigation algorithms [24]. The TRON facility can accurately reproduce a wide range of lighting scenarios representative of the space environment. To mimic the diffused light of Earth’s albedo, ten light boxes are installed around the walls. Each light box consists of a diffuser plate covered with hundreds of light-emitting diodes (LEDs) organized in strips and can adjust their colour and intensity. The light boxes are calibrated to produce radiance across the diffuser plates that are as uniform as possible and compatible with Earth’s albedo in Low Earth Orbits (LEO). A metal halide arc lamp is also used at the facility to simulate direct sunlight. As the background material, light-absorbing black commando curtains are placed over all ambient light sources, including the windows and the deactivated light boxes, to enhance the impact of diffused and direct light [12].
2.2. GRALS, Netherlands
The GNC Rendezvous, Approach and Landing Simulator (GRALS) testbed is situated in the Orbital Robotics and GNC Laboratory (ORGL) at the European Space Research and Technology Centre (ESTEC) [16]. A Prosilica GC2450 camera mounted on a KUKA robotic arm is used for capturing images at the facility. To recreate a realistic space environment from an illumination standpoint, a movable lamp is mounted on a UR-5 robot and directed towards the target mockup during image acquisition. The lamp is a dimmable, uniform, and collimated light source with a spectral response close to 6000 K and an exclusive optical lens which provides high uniformity (±5%) shadow-free backlight illumination. Besides, black background curtains are placed around the robots’ workspace in order to mask most of the background noise, such as unwanted reflections from the robots’ rails.
2.3. EPOS, Germany
The European Proximity Operations Simulator (EPOS) test facility was developed by the German Aerospace Center (DLR) to study rendezvous and docking scenarios [10]. Two different types of cameras are used at the facility: two Charge Couple Device (CCD) Prosilica GC-655 M cameras for capturing intensity images (in the visible spectrum) and two Photonic Mixer Device (PMD) cameras (PMDtec Camcube 3.0 and Bluetechnix Argos3D-IRS1020 DLR Prototype) for capturing depth images. For simulating realistic illumination, an ARRI Max 18/12 theatre spotlight is used [25]. This daylight spotlight is equipped with a hydrargyrum medium-arc iodide (HMI) light source and can generate spectrally realistic irradiation, resembling that in the Earth’s orbit around the Sun. The spotlight is mounted on a 2-DOF yoke that is electrically steerable for easy fine-tuning of the illumination direction. To capture images of the target satellite with a space-realistic background, black curtains are used as background, and the robotic arm carrying the satellite mockup is wrapped with black Molton material.
2.4. ASTROS, USA
The Autonomous Spacecraft Testing of Robotic Operations in Space (ASTROS) platform at the Georgia Institute of Technology supports experiments in vision-based autonomous rendezvous and docking, with a focus on on-orbit servicing of spacecraft [11]. The platform is equipped with a monocular PointGrey Flea3 camera to capture images (and videos) in different resolutions. The facility is capable of producing realistic images by various configurations of lighting and dark environments and can replicate the harsh contrasts of imaging highly reflective surfaces against a dark background as seen in space. However, the technical details of the equipment used for creating these different illumination conditions are not available publicly. An overhead projector on the ceiling projects virtual images from Earth orbit against a projection screen on the wall, [26], which serves as the background for images captured.
2.5. ORION, USA
The Florida Institute of Technology developed the Orbital Robotic Interaction, On-orbit Servicing, and Navigation (ORION) laboratory to test spacecraft GNC systems for proximity manoeuvres and autonomous or telerobotic capture [27]. The ORION simulator uses the commercial-off-the-shelf Litepanels Hilio D12 LED panel to generate a light source sufficiently bright to exceed the dynamic range of common optical sensors while providing a narrow beam angle. The panel generates light with a colour temperature of 5600 K (daylight balanced) with 350 W of power. The intensity can be continuously dimmed from 100% to 0%, and the beam angle can be varied between 10° and 60° using lens inserts. The light can be used not only to simulate solar illumination but also the weaker and diffused Earth’s albedo. The background walls, floor, and ceiling of the testbed are painted a low-reflectivity black paint, and all windows are covered with black-out blinds to fully control the lighting conditions and to reproduce orbital conditions.
2.6. INVERITAS, Germany
Innovative Technologies for Relative Navigation and Capture of Mobile Autonomous Systems (INVERITAS) facility at the Robotics Innovation Center of the German Research Center for Artificial Intelligence (RIC DFKI), designed and constructed under the INVERITAS project, models rendezvous and capture manoeuvres between a client satellite and a servicer satellite in Earth orbit [28]. The facility is equipped with six mobile spotlights to reproduce space-like illumination conditions. Each spotlight is motorized, allowing pan and tilt rotations, and the field of view can be varied between 12° and 30°. The spotlights can be moved up and down from 1-6 m. The 575 W gas discharge lamps used at the facility deliver a 6000 K light, with a maximal intensity of 14500 Lux at a 10 m distance and a 12° field of view. Special light-absorbing paints are used on the background walls, ceiling, and all visible components of the system providing a space-like nonreflective background.
2.7. ARGOS, Italy
The Advanced Robotics & GNC Optical-based Simulator (ARGOS) is a GNC experimental facility at Politecnico of Milano, at the premises of the Aerospace Science and Technology Department (DAER) [29]. ARGOS facility is equipped with satellites and asteroids mock-ups for proximity operations reproduction. The camera is a Chameleon 3 by FLIR, with a resolution up to pixels, a maximum frame rate of , and equipped with a lens having a field of view of deg and a fixed focal length of mm. The facility has been realized in a dedicated dark room, preventing light reflection with black curtains and floor covers. To simulate the sun illumination, a dedicated high-CRI LED array with dimension , deg beam angle and K light temperature is employed [29].
2.8. GMV’s Platform-Art, Spain
The Advanced Robotic Testbed for Orbital and Planetary System and Operations Testing (platform-art), located at GMV’s headquarters in Madrid, Spain, is a dynamic test bench for supporting and enhancing the validation of space GNC technologies and related metrology equipment, with real air-to-air metrology dynamic stimulation [30, 31]. It allows the use of sensor measurements in open and closed loops, through the recreation of relative (full or scaled) trajectory and attitude profiles by using robotic arms. Realistic optical sensing conditions are achieved by using black curtains fully covering the walls and ceiling, achieving a real black environment simulating outer space. Additionally, a single strong light source is used to provide sun-like illumination conditions. [30, 31].
The reviewed literature about vision-based laboratories for space applications presents a summary of the facilities and the image acquisition equipment used. However, details about the different materials and equipment options considered during laboratory development are missing. The logic behind their equipment selection is missing from the literature. Moreover, the fact that every reviewed facility has been developed with different materials and equipment suggests that many commercially available alternatives can be considered to attain similar or equivalent objectives. In addition, each lab development project will probably have different scopes and limitations regarding budget, space available, intended applications, and other resources. Hence, it is interesting to understand the benefits and drawbacks of the different market-available equipment options in terms of cost, ability to emulate a space-like environment, quality of images captured, and other relevant factors. The following section presents a detailed market survey with comparison metrics for commercially available materials and equipment for a vision-based space application lab.
3. Survey of Materials and Equipment
This section reviews commercially available equipment required for image acquisition at a space lab, as illustrated schematically in Figure 1. In this review, background materials (Section 3.1), cameras (Section 3.2), and illumination lamps (Section 3.3) to recreate high-fidelity space conditions are included. The reference links for each item reviewed are given in supplementary material Section A. Note that the candidates selected for this survey are based on the availability and product costs while setting up the SnT Zero-G Lab. The proposed survey metrics can be extended to materials and equipment based on regional availability when building similar labs in future.
3.1. Background Materials
The surveyed background materials are selected to include three diverse categories of materials: fabric, paint, and paper, which are commonly used as background materials in different applications (for example, black velvet fabric material used in digital photography). Table 1 presents a summary of their properties including specification, composition, reflectivity parameters, and unit costs of each item. The reflectivity parameter here refers to the measure of the ability of a surface to reflect radiation. Based on the survey, the reflectivity values suggest that the best commercially available option for space conditions recreation is the fabric black velvet, from KOYO, with the lowest reflectivity of all surveyed materials, 0.1%. However, there are also materials in the market whose reflectivity specification values are not readily available from the manufacturer and thus making a direct comparison difficult. Hence, in Section 5, we propose an experimental analysis to complement the market survey for evaluating the suitability of materials for background in a space lab.
3.2. Cameras
For image capturing at a space lab facility, eight different cameras were surveyed. These cameras were selected to represent a wide cost range (from ~25 to ~3200 EUR) for cameras with different image capture capabilities and applications. Sony A7RIII and Cannon 5DSR are cameras used for digital photography; the C3D CubeSat Camera and NanoCam C1U are specifically designed to be deployed in space, especially the latter one has 3 years of proven flight heritage. The remaining four (Intel RealSense, FLIR Blackfly, Pi cameras (HQ and LQ)) represent the commonly used cameras for robotics and machine learning applications. The results from the survey are presented in Table 2. However, the cameras need to be further compared in space-representative situations (varying illumination and exposure conditions). Hence, we propose additional experiments in Section 6 for comparison of the cameras in such conditions. The market survey along with the experiments provided a comprehensive comparison of camera systems for a space lab.
3.3. Illumination Lamps
In Table 3, a survey of commercially available illumination lamps is presented, selected to represent low-, medium-, and high-cost alternatives. The lamps are assessed regarding their light source temperature/wavelength, luminous flux, power and efficiency, emission angle, total lamp dimensions, cost, and brand. The low-cost alternatives include aluminum reflectors of small (10-20 EUR) and medium sizes (20-60 EUR) of around 15 W that can be bought at regular home goods stores. A medium-cost alternative is proposed with an omnidirectional growth lamp (Low glare downlight, 580-690 EUR) with power up to 276 W. The most expensive alternative included in the survey is the large-area solar simulator (Sunbrick, up to 30810 EUR), with programmable spectra and power up to 625 W.
The section presented a detailed survey of different background materials, cameras, and illumination lamps available in the market. However, this market survey alone is inadequate to make the equipment selection. For instance, the manufacturer may not always provide the needed metrics (as in the case with the background materials), or the equipment may need to operate in a different (harsh space-like) environment than its normal operating conditions (as for the cameras). Hence, to further reduce uncertainty, the survey is complemented with experimental analyses of the background materials and cameras. The laboratory setup for the experiments is described in the next section, followed by experimental analyses of background materials (Section 5) and cameras (Section 6), respectively. The market survey, along with these supporting experimental analyses, will provide information to guide the selection of suitable image acquisition equipment in a space lab.
4. Laboratory Setup
The data collection activities for the experiments presented in this article were conducted at the SnT Zero-G Lab facility. The SnT Zero-G Lab is a multipurpose facility capable of emulating a large variety of in-orbit operations in different orbital scenarios. The facility has two UR10e robotic arms mounted on rails, providing a 6 + 1 DoF. The robotic arms are capable of mimicking the orbital trajectories of the spacecraft, other orbital objects or the light sources. To recreate the challenging lighting conditions in space, the Zero-G Lab uses a Godox SL-60 LED video light. Godox SL-60 has a colour temperature of 5600 K (see Table 3), which is similar to that of the spectrum of the sunlight in orbit [32]. This ensures a similar radiation spectrum in the lab as that in the orbit. Also, the lamp’s power range of 60-200 W and luminous flux of 4500 lumens offer more than adequate illumination for our orbital lab, given its dimensions of less than 5 m in length. Moreover, the SL-60’s design can accommodate a broad selection of 3rd party modifiers, enhancing its ability to mimic different orbital illumination conditions. Finally, due to the compact dimensions, measuring mm, it can be seamlessly integrated into the lab, avoiding unnecessary spatial obstruction. The wall and ceiling are painted black, and epoxy flooring (black) is usually covered with a layer of nonreflecting foam sheet surface to remove reflections and create a space-like environment. The general setup of the laboratory environment for the experiments is shown in Figure 2 and consists of the following components: (i)Cameras. The cameras were mounted on a tripod directly facing the object of interest, i.e., the spacecraft(ii)Spacecraft. The spacecraft was mounted on a UR10e robotic arm. A 1 U CubeSat was used in the experiments presented in this article. However, the experiments can also be conducted using other types of spacecraft mock-ups available(iii)Background. A dark background was placed behind the CubeSat, either mounted on a tripod-like structure or placed independently(iv)Light source. A single continuous light source was mounted on a second UR10e robotic arm using a custom-designed metal bracket fabricated in stainless steel
The experiment setup was designed taking into consideration constraints including the size of the room ( m), visual light (VL) reflective surfaces present like the robotic rails, camera capability limitations, and the physical restrictions related to the range of possible motion for the robotic arms. For all of the experiments, the camera position remains static, while the robotic arms controlled the light source and CubeSat positions. The trajectories/positions for the robotic arms were provided as a set of manually defined waypoints. Python scripts were used for automating the image capture process with different camera settings to determine the appropriate white balance gain parameters and to avoid unwanted color shifts within images for each camera used. Black background materials were placed at a distance of 56 cm to the rear of the vertical center of the CubeSat, and the cameras were placed at a distance of 140 cm directly in front of the vertical center of the CubeSat.
4.1. Cameras
In the experiments presented, two different cameras were investigated: (1) a Raspberry Pi low-quality (LQ) camera, and (2) a high-quality (HQ) camera. Technical specifications of the cameras are given in Table 4. The LQ camera relies on an inbuilt lens, whereas the HQ camera uses a 12 mm Edmond Optics lens.
Raspberry Pi cameras are low-cost, widely available, work seamlessly with Raspberry Pi computers, are easy to control through Linux-based programming scripts, and have proven space heritage. The Demonstration of Technology (DoT-1) satellite mission was the first to image and video the Earth with a commercial-grade, off-the-shelf Raspberry Pi camera [33]. Similarly, in 2015, the Astro Pi project included a Raspberry Pi computer and camera in its payload [34]. British ESA astronaut Tim Peake used the Raspberry Pi equipment to conduct a series of educational experiments on board the International Space Station [35]. Since then, subsequent Astro Pi projects have been launched in 2021 and 2022 with upgraded payloads, including the Hi-Quality Raspberry Pi camera such as the one investigated in this work [36]. Importantly, NASA has created an open-source Flight Software & Embedded Systems Framework, F’(F Prime), composed of flight-worthy components for small-scale spaceflight systems [37]. With its Linux compatibility, Raspberry Pi is included as one of two examples F’ deployments developed with F’ [38]. NASA has also established guidelines for employing Raspberry Pi products in space [39]. The choice to focus on the Raspberry Pi cameras is justified in the current investigation given the wide availability of budget-friendly Raspberry Pi cameras with their proven space heritage, in addition to the access to reliable development tools. Note that the experimental analysis presented in this paper is not limited to the selected cameras but can be extended to camera models available at the time of construction of similar labs in future.
For all of the experiments conducted, the reference camera position remained fixed and is denoted as CP0. In Figure 3, CP0 is illustrated with the horizontal angle of 0 deg and the vertical position labelled s: straight on, indicating the camera was looking straight on the CubeSat at its central height.
(a)
(b)
4.2. Lighting
All experiments were conducted with a single light source (Godox SL-60 LED Video Light) available at the SnT Zero-G Lab facility. The technical specifications are given in Table 3. Three lamp configurations were investigated, mimicking various illumination conditions from a space environment. For example, collimators, produce parallel light beams that create hard shadows and large differences in light intensity between illuminated and dark regions are typically chosen for mimicking objects in space illuminated by the sun without an atmosphere [28]. The lamp configurations used in the experiments are defined below: (i)LAMP0. Light source with a collimator as light modifier(ii)LAMP1. Light source with a reflector as light modifier(iii)LAMP2. Bare light source without any modifiers
Additionally, two different light intensities were also used for each of the lamp configurations: (i)Light Intensity Low (LIL). 10% of the total light intensity available from the source(ii)Light Intensity High (LIH). 100% of the total light intensity available from the source
The combination provided six () different illumination conditions used in the experiments. The reference lighting position (LP0) was defined as 30 deg to the right of the camera in the vertical plane and above the camera height (a: angle down) horizontally, as shown in Figure 3. LP0 position enables the visualization of both shadows and highlights of the CubeSat and as such, best represented the 3D structure of the object when projected onto a 2D image plane. Details of other possible lighting positions, along with a qualitative comparison of the corresponding captured images are provided in supplementary material Section B.
5. Background Analysis
The objective of the background materials experiment is to determine the background with the highest light absorption such that it appears featureless in the captured images. The backgrounds analysed in this experiment are black velvet fabric (BG0), Moussu paint (BG1), Black 3.0 paint (BG2), black background-paper (BG3), and Neewer background fabric (BG4). Refer to Table 1 for more details. In the context of the performed experiments, a “featureless background” corresponded to the one that added no discernible information to an image and most closely resembled the black colour, as represented by the RGB pixel value of (0,0,0).
5.1. Data Collection
For the data collection, the camera was set to position CP0, and the background to be tested was placed 196 cm behind CP0. In this case, the CubeSat was not used and was, hence, removed from the camera’s field of view. Images were captured for each of the background materials (BG0-4) under different light intensities (LIL and LIH), lamp configurations (LAMP0-2), and for five angles of illumination (LA0-4) as illustrated in Figure 4. A set of sample images collected is shown in Figure 5. The captured images were cropped manually to include the background region before the experimental analysis.
5.2. Experiment and Results
To assess the backgrounds, a reference image is used to compare all the captured images of each background. The chosen reference image is a synthetically created entirely black image produced by setting the RGB pixel values to (0,0,0). Two evaluation measures are used to quantify the comparison of different backgrounds, namely, Root Mean Square Error (RMSE) and Universal image Quality Index (UQI) [40]. Although the former is the most often used distortion metric, the latter offers much more accurate results than the RMSE. Given and as the input and reference image signals, respectively, RMSE can be formulated as where is the total number of pixels and , are the pixel values of input and reference image, respectively, at the -th pixel location.
Figures 6 and 7 show the comparison of obtained RMSE values from respective backgrounds for LQ and HQ cameras, respectively. Note that the error is computed between each of the captured images, with different background and illumination settings, and the reference image. It is evident that the images captured with black velvet tissue (BG0) have the lowest RMS error both in the high intensity (LIH) and low intensity (LIL) light conditions, which indicates that they are the most similar to the reference image among the other backgrounds.
Despite the wide use of RMSE, the computed score does not truly reflect the underlying meaning of image similarity. Hence, we propose to make use of another more meaningful image similarity index, i.e., UQI. Unlike the traditional error summation methods, UQI considers the following three factors for modelling any image distortion: loss of correlation, luminance distortion, and contrast distortion. Given and as the input and reference image signals, respectively, UQI can be formulated as where , , , and represent the mean and standard deviation of all the input and reference samples, respectively, and , the correlation. In practice, this score is computed in local neighbourhoods using a sliding window, then the local qualities are averaged to obtain the global UQI. To compute a local quality, the following rules hold. If the denominator is nonzero, then since is zero ( is a black picture), the local quality is also zero. However, the denominator is nonzero if and only if the pixel values in both pictures are not all the same, and when they are all the same, then the quality is automatically set to 1. Therefore, local quality values are either 0 or 1, according to the exact similarity (or not) of local windows across the two pictures. Finally, UQI averages these 0 and 1 values, therefore measuring the proportion of local windows that are identical between the two pictures. Moreover, UQI provides error measurements independent of the viewing conditions and individual observers (subjective analysis by humans) [40].
In Figures 8 and 9, the UQI scores for all the acquired images (under different illumination conditions) with respect to the reference image for the LQ and HQ cameras, respectively, are presented. A similar trend can be observed in the results from UQI to that reported using the RMSE. In other words, the black velvet fabric (BG0) had the highest UQI scores compared to all other backgrounds, both in the high-intensity (LIH) and low-intensity (LIL) light conditions.
These results indicate that BG0 is the most featureless background with the highest light absorption, which is in agreement with the market survey (Section 3.1). This performance makes it the best choice, for image acquisition in a space lab. For the rest of the experiments detailed in this article, the BG0 was used.
6. Camera Analysis
Experiments were performed with the two Raspberry Pi cameras (LQ and HQ) with the objective of performing a (i)Qualitative comparison of the ideally exposed images under varying illumination conditions for the LQ and HQ cameras(ii)Quantitative study of the image quality degradation with different exposure settings (over-exposure and under-exposure) for both cameras
Together, these analyses provided an experimental framework for comparing camera capabilities. Furthermore, along with the market survey presented in Section 3.2, it can support in making purchase choices for camera selection in a space lab.
6.1. Ideal Exposure Analysis
Camera exposure settings are defined by three parameters: aperture, shutter speed, and ISO (gain), known as the exposure tuple (A:SS:ISO). The exposure tuple determines a given exposure value (EV); multiple exposure tuples can result in the same EV [41]. Images captured with small values of aperture, for example, f/2, will allow more light to reach an image sensor for fixed shutter speed and ISO settings, which can be a useful property when capturing images in low-light conditions. However, the choice of aperture also impacts the depth of field (DOF) which determines which portions of a 3D object, relative to the focal plane, will appear in focus when projected onto a 2D image plane. The smaller the value of the aperture, the shallower the DOF. Therefore, a trade-off exists between image sharpness and brightness when an aperture setting is chosen. Another consideration is the focal length of the lens employed. The greater the lens focal length, the shallower the DOF for a fixed value of aperture. The shutter speed parameter dictates how long a camera’s image sensor will receive light. Whether an image will be properly exposed is also a function of the shutter speed. The ISO parameter of the exposure setting impacts how sensitive an image sensor is to light. However, more noise will be introduced into an image if the ISO value is set to be more sensitive to light. In the performed experiments, to not introduce unwanted color shifts into images taken with the same camera, “autowhite balance” was disabled and the red and blue gain settings listed in Table 4 were applied for each camera.
6.1.1. Ideal Exposure
A careful choice of the exposure tuple is required to generate an “ideally exposed” image within the context of the given illumination conditions and the mechanical limitations of the image capture device (camera). The concept of ideal exposure is application dependent. In the case of the performed experiments, ideally exposed images were those which had the CubeSat well illuminated with all the features (like edges, corners, and surface panels) clear and distinguishable. To obtain the initial ideal exposure settings under different illumination conditions, a Sekonic L-558R DualMaster light meter was placed directly in front of the spacecraft object. Then, a careful visual inspection of images captured with further fine-tuned exposure settings was used to define the ideal exposure setting for the experiment.
6.1.2. Data Collection
The cameras and the lamp were mounted at their reference positions CP0 and LP0, and the CubeSat was positioned between the background and camera at a distance of 56 cm and 140 cm, respectively, as shown in Figure 2. The images were captured under different light intensities (LIL and LIH) and lamp configurations (LAMP0-2).
6.1.3. Experiment and Results
Ideal exposure settings for images captured under different illumination conditions were obtained, as described with a light meter and by visual inspection. The selected ideal exposure settings with the corresponding images are shown in Figure 10. The qualitative analysis of these images showed that, by carefully selecting the exposure settings, good-quality well-exposed images can be obtained for different quality cameras under varying illumination conditions. This is particularly relevant in space-like environments where well-exposed images with clear and distinguishable features need to be captured under considerable variations in illumination conditions. The results also suggest that even with an LQ camera, images of good quality can be captured if exposure values are well calibrated.
6.2. Exposure and Image Quality Analysis
A reference exposure (EX0) was defined to study the effect of overexposure and underexposure on image quality. An aperture value of f/2.0 was selected as it allowed for an acceptable DOF and was achievable with all camera lenses tested. In addition, an aperture value of f/2.0 made it possible to capture images in a low-light setting at a shutter speed that would not introduce motion blur in the established laboratory setting. The reference shutter speed was set at 1/30th of a second. Finally, the ISO value of 100 was chosen so as not to introduce unwanted noise into the captured images. Thus, the reference exposure for capturing reference exposure images in this experiment is defined with the exposure tuple values of (f/2 : 1/30 : 100).
The reference light intensity (LI0) corresponds to the light intensity required to set the reference exposure EX0 as the ideal exposure for each of the lamp configuration tested. To establish the LI0 light intensity, a light source was placed at the reference light position LP0 as shown in Figure 3. The light intensity was then adjusted until the reference exposure provided ideal exposure. The LI0 values for LAMP0, LAMP1, and LAMP2 configurations were 75%, 25%, and 30%, respectively.
6.2.1. Data Collection
Images were collected under the same positional setup described in Section 6.1. The images were captured only for the reference light intensities (LI0) and for lamp configurations (LAMP0-2) with each of the exposure settings defined in Table 5. In this experiment, the underexposed and overexposed (EO1-EEO) conditions were obtained by changing only the camera shutter speed. The aperture and ISO values were kept the same. The HQ camera captured images with a 12 mm lens, while the LQ camera had an inbuilt 6 mm lens which affects the field of view in captured images. Also, because the cameras were mounted side-by-side, the cameras’ view fields were slightly horizontally translated. Therefore, images were cropped to centrally align the CubeSat prior to analysis.
6.2.2. Experiment and Results
A quantitative analysis of the image quality degradation with varying exposure settings for a given illumination condition (light intensity and lamp configuration) was performed. Since the images contained a single CubeSat object, with a fixed background, the structural similarity with respect to the reference image provided a relevant measure of image quality degradation. The multiscale structural similarity index (MS-SSIM) [42] was used to measure structural similarity. The MS-SSIM was derived from the Structural Similarity Index (SSIM) [43] extending it to incorporate multi-scale measures using image details at different resolutions. MS-SSIM separates the influence of illumination (average luminance and contrast) to explore the structural information in an image.
In Figure 11, the MS-SSIM score for the acquired images under different exposure settings with respect to the reference image is presented. For the camera with a higher sensor quality (dynamic range), the image degradation was slower compared to one with a lower sensor quality. For both the cameras, the structural integrity of the images dropped identically on both sides of the curve (over and under exposed) with respect to the reference image. This behaviour indicates that the relative drop in image quality for both the cameras is similar under extreme exposure settings.
7. Discussion
The results of our experiments suggest that laboratory equipment selection is not a straightforward procedure. This was evidenced in the experiments performed with the Raspberry Pi cameras in Sections 6.1 and 6.2. As shown in Section 6.1, a careful calibration of exposure settings can produce reasonable-quality images for both the LQ and HQ cameras for proper exposure settings. Similarly, Section 6.2 indicated that the relative image degradation in extreme exposure settings for both the cameras is identical, despite their differences in price and other technical specifications. Hence, the camera selection is highly application-dependent and needs to be analysed experimentally, case by case. Market surveys, such as the one presented in Section 3.2, will serve as a starting point and need to be followed by experimental analyses.
The same criteria applies to the selection of the background material. The experimental results indicated that the highest UQI was obtained with black velvet fabric, which was an expected result as this was the material with the lowest VL reflectivity (Section 3.1). However, as different manufacturers might implement different methods to determine the reflectivity of their products (or might not even provide a reflectivity value at all), if possible, UQI (or even VL reflectivity) of dark background materials should also be tested before making a major purchase. The SnT Zero-G Lab facility is still under development; further experiments will be conducted for camera analysis with images captured in scenarios where the space object moves along a trajectory relative to the camera. The objective of these experiments would be to introduce motion blur and other effects common during applications like vision-based navigation. Future work will also focus on conducting an experimental analysis of different lighting sources to support the market survey in this article. The spectral analysis will provide a more accurate comparison of light sources and help to analyse their similarity to real space conditions.
8. Conclusion
Facilities simulating real-world space environments are an integral part of training and validating vision-based space applications. High-fidelity space-like images with annotations can be collected from these facilities to train and test the algorithms. However, the development of such a space lab is challenging. The current literature lacks support in the form of manuals or templates. In this context, this article focused on a key aspect of a space lab facility development, which is the equipment selection. This article presented a systematic approach to equipment selection for the image acquisition process, based on the lessons learned during SnT Zero-G Lab development at the University of Luxembourg. The approach combines a market survey of equipment followed by experimental analysis. Background materials, cameras, and illumination lamps were surveyed. The background materials were first compared based on the VL reflectivity values obtained from the manufacturers. For comparing materials with unknown reflectivity values, we present an experimental analysis method that calculates UQI scores with reference to a synthetically generated black image to identify suitable options. For camera selection, experiments suggest that a market survey alone will not provide sufficient information to make a purchase decision. The results demonstrate that it is possible to obtain comparably good-quality images even from a lower-quality (less expensive) camera by carefully calibrating the exposure settings. Hence, for selecting camera systems, the market survey and experimental analysis should be used in tandem to gather the required information. Future work is planned for studying image quality when motion blur and other phenomena are introduced and for studying the performance of different light sources for simulating the space environment.
Data Availability
The datasets used for this study will be made available upon request. Requests are to be sent to the corresponding author.
Disclosure
The authors acknowledge that earlier versions of the paper are publicly available as preprints in ArXiv [44] and DeepAI [45] websites.
Conflicts of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Acknowledgments
This work was funded by the SnT—University of Luxembourg’s internal funding for the project “Zero-G Lab—Multi Purpose Zero Gravity Lab Facility” and by the Luxembourg National Research Fund (FNR), grant reference BRIDGES2020/IS/14755859/MEET-A/Aouada. For the purpose of open access and in fulfilment of the obligations arising from the grant agreement, the author has applied a Creative Commons Attribution 4.0 International (CC BY 4.0) license to any author-accepted manuscript version arising from this submission.
Supplementary Materials
Section A gives the reference links for each item reviewed in the market survey. In Section B, the details of different possible lighting positions that can be used in the experiments, along with a qualitative comparison of the corresponding captured images, are provided. (Supplementary Materials)