Built On-Orbit Robotically Assembled Gigatruss (BORG): Ground Robotic Demonstration

Abstract

The next generation of large space infrastructure will require crucial advancements in current technology. Current methodologies focus on large deployable structures folded into cramped payload fairings or revolutionary assembly techniques requiring many moving components. Utilizing both in-space assembly and deployable concepts, a hybrid mixed assembly scheme was posed using smaller deployable units interspersed with rigid connecting elements to assemble these large architectures. The Built On-Orbit Robotically Assembled Gigatruss (BORG) structure allows for modularity in assembly and repair with the number of separate elements comprising the structure to be reduced, compared to strut-by-strut assembly. The following documents the process of constructing and running physical trials on a prototype BORG architecture. Additionally, a Semantic and Fiducial Aided Graph Simultaneous Localization and Mapping (SF-GraphSLAM) approach is taken to verify the relation of assembled and deployed truss elements to aid in error evaluation and state estimation. This technology demonstration stands as a proof of concept in verifying the viability of the BORG architecture as a method for large structure assembly.

1. Introduction

With the rise in upcoming missions to space, a resurgence citing the critical need for large space structures has made its way to the forefront. The backbone structure required for these architectures is the space truss framework, which will remain the primary focus of the work. The current standard for large structures successfully launched into space has employed the use of complex deployable maneuvers, such as the James Webb Space Telescope. A paradigm shift to in-space servicing, assembly, and manufacturing (ISAM) addresses many scalability and reliability factors critiqued within the deployable scheme but brings along its own challenges. One of these challenges arises in the strut-by-strut scheme, where each longeron is individually assembled, resulting in a very high number of elements to be placed. As the structure is scaled up, this becomes increasingly less practical.

In the paper “Built On-orbit Robotically-assembled Gigatruss (BORG): Mixed Assembly Architecture Trade Study” [1], a hybrid mixed assembly scheme is posed, using smaller deployable units interspersed with rigid connecting elements to assemble large architectures. This methodology allows for a square bay truss structure to be constructed out to nxnxn number of bays. This is accomplished by alternating the previously mentioned deployable bays with closeout elements in an alternating checkerboard pattern. This allows for the number of separate elements comprising the structure to be reduced and provides an avenue for replacing these elements and allowing for future re-configurations. The trade study proves that this assembly approach has merit to be considered for implementation on the same level as the strut-by-strut ISAM methodology.

The following research covers the preliminary testing that occurred as the BORG methodology transitioned from concept to physical hardware. Many of the design choices laid out within this documentation could be changed to adapt to specific needs. This prototype structure outlined below focuses on low-cost hardware and creating a ground demonstration environment for rapid in-space truss assembly testing.

2. Background

Between both in-space and ground demonstrations, there exists a history of experiments advancing the field of in-space assembly and inspection covered in the ISAM State of Play [2]. This research covers a wide range of architectures; however, trusses exist as an extremely prevalent assembly drive. Past research such as that seen in the Automated Structures Assembly Laboratory (ASAL) [3] and the NASA Intelligent Jigging and Assembly Robot (NINJAR) [4] show ground demonstrations of a space truss structure featuring strut-by-strut insertions.

Research has been conducted on the assembly of deployed modular components and has also been conducted on the application to the ISAM scene. Seen in both Robotically Assembled, Modular Space Telescope (RAMST) [5] and Precision Assembled Space Structure (PASS) [6], this architectural style is often applied to a telescope mirror assembly endeavor. In a series of Robosimian trials [7], modular units were also autonomously assembled, utilizing deployable elements with AprilTags for navigation. In each of these cases, the modular units feature identical units which are joined together.

Following successful ground demonstrations, GITAI sent their S1 [8] robotic assembly agent to the International Space Station for a critical series of technology demonstrations. This also featured the use of visual fiducials to aid in the autonomous assembly tasks. With this successful execution, S2 has recently been launched to ISS to continue advanced testing.

The ability to remotely inspect spacecraft and satellites was identified as a critical and rapidly approaching ISAM capability in a recent global trends study [9]. Current technologies within the inspection sphere exist as an intersection of two categorizations [2]. The first of these focuses on the attachment point of the inspection agent as either a free flyer or anchored vehicle, while the other focuses on the inspection style being visual or not visual. The vast majority of these technologies skew towards a visual inspection style, reinforcing its importance. A recent critical space demonstration of this technology can be seen with the AeroCube inspection missions, as they performed satellite-to-satellite visual inspection [10].

The following research builds off this common foundation of truss elements being assembled in a ground demonstration with validation with a new architecture style and a unique SLAM visual inspection procedure.

3. Materials and Methods

3.1. Mixed Assembly Method

This section contains a summary of the findings and concepts further explored in the paper “Built On-orbit Robotically-assembled Gigatruss (BORG): Mixed Assembly Architecture Trade Study” [1]. In implementing the use of large space structures, there currently exist several methodologies. These focus primarily on the deployment of completed hardware units, such as the James Webb Space Telescope (JWST) [11], or on the assembly of individual structural components [4]. Each of these solutions provides unique challenges to the problem. Single-unit structures with many deployment steps are size-constrained based on their packing efficiency and the payload fairing of the selected launch vehicle. This also introduces a single-point failure condition in which any single deployment step failure could cost the mission success. In response to these challenges, ISAM components were developed. By utilizing many elements, any number of these components could be replaced throughout the mission life cycle with components delivered by much smaller rockets. The trade off in this, however, is the utilization of so many individual components. These components need to be individually tracked and verified throughout the very lengthy assembly procedure. To strike a balance between these two solutions, a hybrid mixed assembly approach was conceived. This features small, contained deployable units that are held together with both close-out-struts and close-out-squares. By doing this, the modularity, launch advantages, and replacement protocols from the strut-by-strut architecture remain present while drastically limiting the number of required elements especially as the complexity of the structure increases.

3.2. Hardware

In order to test the application of the mixed assembly BORG scheme, physical trial plans and the required hardware were critical. The components required to achieve this included the hardware of the truss modules, the robots for assembly, supporting platform elements, and computer vision and simultaneous localization and mapping (SLAM) techniques to analyze the structure. This BORG truss testbed developed in the FASER Lab allows for further research into robotic in-space assembly and the servicing of truss structures.

3.2.1. BORG Truss Module Design

Upon approaching the overall design factors for the BORG truss, certain features had to be considered. The primary goal of the mixed assembly scheme is to feature deployable elements with rigid elements connecting them. This scheme is repeatable out to any odd number of trusses on a single side, with three being the smallest number of viable units. Following this, the BORG truss example used for this proof-of-concept experimentation was conceptualized as the 3 × 3 × 3 truss shown below in Figure 1.

This mixed assembly method utilizes three types of modules to create the 3 × 3 × 3 BORG structure: (1) 9 deployable trusses, (2) 6 close-out-squares, and (3) 12 close-out-struts are shown in Figure 2. Note that there are some variations in the deployable truss corner nodes depending on whether it is an exterior edge truss or the center truss of the 3 × 3 × 3 example. This is extremely tied to the individual nodal geometry utilized in this testbed and is dependent on the specific nodal interaction features in a given design.

The deployable truss has flexible, pre-tensioned cross diagonals made of metal cabling. The deployable truss is designed with corner nodes that include a one-way bearing to allow it to fully deploy but not back drive. The inner anatomy of the corner nodes and mid-nodes of the deployable strut are shown below in Figure 3.

Figure 4 shows the element interaction between a node and a captured strut or square. The capture channel features a spring active plunger that pins the insertion element in place through a corresponding groove. This allows the insertion to slide into the capture window and then be locked in place throughout the remainder of the assembly operations. The insertion elements can be removed with significant force or by backing out the plunger in between assembly trials.

3.2.2. BORG Truss Module Prototype

These test truss modules were manufactured and assembled in the FASER Lab. The corner nodes, deployable mid-nodes, strut ends, and close-out-square corners were 3D printed with PLA filament. The deployable corner nodes had the internal components for deployment, shown in dotted lines in Figure 3a, pre-inserted before the node was clamped inside a concave custom clamp while the epoxy cured. The flexible diagonal components were comprised of two flexible sections per side with a turnbuckle placed in the middle. The connections to both the turnbuckle and inner node components were facilitated by a small crimped loop of cable. This was ideal for separating the top and bottom planes of the deployable, overcoming any small measurement inconsistencies, and fine-tuning the pre-tensioning values. A jig was constructed to ensure the flexible diagonal lengths were consistent to create the desired pre-tension.

The struts were constructed with 0.875 inch diameter aluminum tubing and epoxied to the 3D printed elements. The deployable trusses were assembled using a jig to hold the nodes 0.5 m apart during the curing process. Jigs were also used to make sure the half-strut and full-strut ends were adhered with the correct pivot point-to-pivot point distance. The close-out-square corners had another jig to ensure that that module was assembled properly. Some hardware assembly photos are shown in Figure 5 with the final units shown in Figure 6.

3.2.3. Robotic Manipulators

The robotic manipulators utilized in this experimentation the Lightweight Surface Manipulation System (LSMS), originally developed at the NASA Langley Research Center. LSMS is used for gross manipulation such as the deployment and placing of modules. These operations are aided by a dexterous manipulator, the Jigging Apparatus for Closeout Structures (JACS), and an end-effector Stewart platform (SP) robot, used to help with precise adjustments to refine the deployment and placement of modules. In addition, JACS has an onboard camera for local metrology.

The SP used for this testing, shown in Figure 7b, is constructed of six Actuonix model P16-P LAs, with feedback and 200 mm maximum extension. Each LA is made up of a shaft connected to the top plate and a motor connected to the bottom plate. These top and bottom plates are both water jetted out of quarter-inch-thick aluminum. The ends of the LAs are connected to the plates with an oil-embedded ball joint with a maximum ball swivel angle of 50 degrees. The other end of the ball joint is connected to the plates using a right-angle bracket. Load cells in line with the actuators are threaded directly into the end of the LA shaft and connected to the top ball joint using a thread adapter. JACS is outfitted with several interaction features for the use in robotic assembly tasks. The first is the right-angle interface between the LSMS and JACS. The second is a lifting hook that grabs onto the top bar of the deployment unit to both transport it and deploy it to its full height. Additional gripping features were planned for the implementation on JACS; however, at this stage, they are not included in this demonstration.

3.2.4. Camera

The camera used in this testing is the Luxonis OAK-D camera [12], shown in Figure 8. It contains three cameras and an inertial measurement unit (IMU). For testing the right camera, video was used for analysis. The specifications of that camera is it uses a OV9282 grayscale sensor and has 81° DFOV, 72° HFOV, 49° VFOV, a resolution of 1MP (1280 × 800), focus of FF: 19.6 cm—infinity, and a max frame rate of 120 FPS, and the shutter type is global.

3.2.5. Turntable

An electronic turntable was placed under two sheets of connected plywood with casters evenly spaced out on the edges. Node cup holders were 3D printed and fixed in an array 0.5 m apart, on top of wooden dowel rod offsets to allow for ample maneuvering space for future node robotic grasping tests. This turntable assembly is shown in Figure 9. For this testing, the turntable was manually powered on and off to perform rotations during assembly and inspection, but this process could be integrated with robotic assembly controls to allow for more automation.

3.2.6. OptiTrack Motion Capture System

An OptiTrack motion capture system was used as another set of data points to track the position and orientations of truss modules, the camera, and turntable. For this, fourteen Prime^x 13 cameras from Optitrack (NaturalPoint, Corvallis, OR, USA) [13] seen in Figure 10a were mounted on tripods and partitions surrounding the central testing space. The camera operated at 240 FPS with a +/− 0.2 mm 3D accuracy [14]. Object pose data were recorded at a frame rate of 50 FPS. The motion was tracked by detecting the reflected light off of retroreflective markers placed onto objects which are circled in Figure 10b. By placing four or more retroreflective markers onto an object, it can be made into a rigid body and tracked throughout the scene, shown in Figure 10c, along with any other known objects.

3.2.7. Metrology Marker Unit

A hybrid AprilTag and OptiTrack metrology marker was designed for this testing, shown in Figure 11 on an example deployable truss. The two methods of measurement were combined in a single metrology unit so that known correlations between the AprilTags were used to calculate guesses for the module positions in the optimized SF-GraphSLAM algorithm while also having OptiTrack markers serve as a comparison to the real-world measurements due to expected hardware error. Each metrology marker had a unique AprilTag and OptiTrack four retroreflective marker pattern to allow them to be distinguished. The metrology markers were manufactured with laser-cut wood frames and 3D printed brackets to mount them to the truss modules. A jig was used when attaching them to the modules to ensure that the relation between the adjacent node and the AprilTag was uniform. Note that the same design was also used for close-out-strut and close-out-squares. A slightly modified design was used for mounting markers to the turntable, and the corner units needed two AprilTags.

Figure 12 below shows the numbering system for the modules based on the assembly order, as well as an example of the corresponding AprilTag numbering order.

3.2.8. Test Setup

The FASER Lab facilities were utilized to complete this demonstration with the following key elements shown in Figure 13. This setup features the LSMS grounded in the back corner of the workspace to provide a maximum operational zone as the long-reach manipulator in the scene. The Stewart Platform style JACS is mounted on the end of the LSMS for smaller manipulation tasks as well as serving as a mounting place for the cameras. A raised workspace platform contains the entirety of the planned manipulation space. On this platform sits the turntable for the assembly site as well as the storage location of all un-assembled BORG components. At the storage location also sits the deployment area as well. This deployment site consists of a plate that the lower bar of the deployable module hooks underneath. In lieu of being mounted to the floor this deployment mechanism is weighted down to allow for the proper counterforce for deployment. In a finalized version, a counterforce mechanism at the truss storage location would replace this feature. For example, if the trusses are stored in a canister-like launch vehicle fairing, there could be a mechanism at the opening that allows the top of the deployable to be pulled out and then grabs onto the bottom of the deployable to allow the robot to fully deploy the truss before releasing it. Finally, the entire space is walled off with paneling that the Optitrack cameras are mounted to as they survey the trials.

3.3. Software

The core analysis software used for this testing is based on the Simultaneous Localization and Mapping (SLAM) development performed in parallel for another journal article called “Semantic and Fiducial Aided Graph Simultaneous Localization and Mapping (SF-GraphSLAM) for Robotic In-Space Assembly and Servicing of Large Truss Structures” [15]. The SF-GraphSLAM approach was created using simulation testing in the other journal and this journal article focuses on applying it to physical hardware testing.

SF-GraphSLAM Review

SF-GraphSLAM is built on the concept that using the mixed assembly method, sparely placed fiducials can be used to minimize the state vector needed to describe the BORG truss structure. Furthermore, semantic information of the relations between AprilTags, depending on the deployable mechanisms and assembly modules, can be used to further improve estimations. The SF-GraphSLAM algorithm can be run after each assembly step to verify that it was completed properly and that the deployable or assembled components are within the desired parameters. It can also return the error of the assembled truss compared to the ideal truss. SF-GraphSLAM was built by adding semantic connections between map elements, based on known relations between them, based on the ideal truss structure, to a base GraphSLAM approach [16]. GraphSLAM [16] is also used as a comparison for some of the results.

During the testing, data were collected from two sources: (1) the on-board camera and (2) the global OptiTrack system. Post-processing software, adapted from AprilTag3 [17] and pupil-apriltags [18], was used to run the recorded video from the on-board camera and record the AprilTags observed at each time step with their relative position and orientation. The camera also had an inertial measurement unit (IMU) which records data. The OptiTrack recorded .csv files of the position and orientation of all the markers. The individual markers were grouped together for each metrology unit, and a centralized origin was used to track the unit. The OptiTrack data was extracted from the saved trial run .csv files, building off of [19]. The post-processing software, building off of Basic Robotics [20], was also responsible for syncing up the time of the AprilTag detections, IMU data, and OptiTrack data to return a data set only when all three were overlapping and bucketed into time stamps to compensate for different frame rates. Note there are various coordinate frames for each of the components, shown in Figure 14.

The processed camera movements and AprilTag measurement testing data were then run through the SF-GraphSLAM algorithm and the guess for the state vector of all the camera poses and AprilTag poses was optimized. These guesses were then compared to the OptiTrack data for the respective AprilTags. Note that a unique transform was also applied for each metrology unit from the origin to the AprilTag center. This offset was based on the positions of the sub-markers since their relative positions change the tracked origin location.

4. Results

4.1. Trying Point Cloud Based VSLAM

In order to obtain a reference for how a SLAM with no AprilTag measurement data and no semantic knowledge of the structure would perform a test using an open source visual SLAM (VSLAM), called Stella VSLAM, ref. [21] was run. The OAK-D camera (Luxonis, Denver, CO, USA) was used for this test and was walked around layer 1 of the BORG truss. Figure 15 shows the output of that test.

Conducting this test reaffirmed the choice to focus on AprilTag detection for these hardware tests. Feature extraction from a point cloud-based SLAM would have been a lot less accurate than the AprilTag returns. It would have been feasible to try another computer vision method more tailored to finding spheres or rectangles to try to work with the anatomy of the truss structure, but there seems to be too much noise from these measurements for SF-GraphSLAM for this to have been able to work.

4.2. BORG Manufacturing Error Analysis

To try to quantify manufacturing error, a deployable module was measured in a fully deployed state prior to testing to see how close to the ideal transform the hardware module could come. To test this, both OptiTrack and AprilTag measurements were used. GraphSLAM and SF-GraphSLAM were run using the AprilTag measurements and compared against the OptiTrack data and raw AprilTag measurements to see if the estimated distances between the top and bottom plane tags aligned for all. Figure 16 shows the setup of the truss 2 used for the calibration test example. This is the view from the camera, and coordinate frames for the camera and AprilTags are shown. Figure 17 shows the results of running GraphSLAM and SF-GraphSLAM on this data set. The results of this test are included in

Table 1. Comparing AprilTag measurements, OptiTrack references, and GraphSLAM and SF-GraphSLAM pose estimations for Tag_3 and Tag_4 of the example calibration deployable.

Category	Identifier	X_Trans (m)	Y_Trans (m)	Z_Trans (m)	X_Rot (rad)	Y_Rot (rad)	Z_Rot (rad)
Tag 3 Pose Estimate	GraphSLAM	−0.03341176	0.23270883	0.77072598	0	0	0
	SF-GraphSLAM	−0.03341176	0.23270883	0.77072598	0	0	0
	Raw AprilTag Measurement at t = 0	−0.033615	0.236615	0.780796	0.996213	−0.008484	−0.066907
	OptiTrack Reference	−3.367959	0.339124	0.847187	0.10466	0.811889	0.071459
Tag 4 Pose Estimate	GraphSLAM	−0.02597586	−0.262368	0.72929555	0	0	0
	SF-GraphSLAM	−0.02597586	−0.262368	0.72929555	0	0	0
	Raw AprilTag Measurement at t = 0	−0.025706	−0.262339	0.731435	0.993234	−0.007135	−0.113714
	OptiTrack Reference	−3.359789	0.840594	0.858209	0.019802	−0.057923	0.009992
Tag 4 wrt Tag 3 Transform	Ideal Simulation	0	0.5	0	0	0	0
	Raw AprilTag Measurement at t = 0	0.039098	−0.311234	0.391228	−0.002534	−0.020398	−0.040027
	GraphSLAM	0.007436	−0.495077	−0.04143	0	0	0
	SF-GraphSLAM	0	0.5	0	0	0	0
Absolute Value Difference from Ideal	GraphSLAM	0.007436	−0.004923	0.04143	0	0	0
	SF-GraphSLAM	0.007436	−0.004923	0.04143	0	0	0
Category	Identifier	Distance (m)
Distance of Tag 4 to Tag 3	Ideal	0.5
	OptiTrack Reference	0.501656964
	Raw AprilTag Measurement at t = 0	0.501452828
	GraphSLAM	0.496863002
	SF-GraphSLAM	0.499999777
Difference of Distance	GraphSLAM vs. OptiTrack	0.004793961
	GraphSLAM vs. Ideal	0.003136998
	SF-GraphSLAM vs. OptiTrack	0.001657187
	SF-GraphSLAM vs. Ideal	0.000000223

, showing that all measurement methods estimate the distance of the top and bottom tags to be approximately the ideal 0.5 m.

4.3. BORG Full Truss Assembly Testing

For the hardware testing, each module was assembled in the order shown in Figure 12a and described in the

Table 2. BORG full truss assembly steps. Steps in bold were completed robotically in addition to inspection tasks between each step.

Step	Assembly Task(s)	Step	Assembly Task(s)
1	Place 1P	9	Deploy and Place 13D, Insert 14S
2	Deploy 2D and Place	10	Insert 15P and 16S
3	Deploy and Place 3D	11	Deploy and Place 17D, Insert 18S
4	Deploy 4D and Place	12	Insert 19P and 20S
5	Deploy and Place 5D	13	Deploy and Place 21D, Insert 22S
6	Insert 6S, 7S, 8S, and 9S	14	Insert 23P and 24S
7	Deploy 10CT and Place	15	Insert 25P and 26S
8	Deploy and Place 11D, Insert 12S	16	Insert 27P

below. Letters are used to signify what type of module is being assembled: D = Deployable, CT = Center Truss, P = Close-Out-Panel, S = Close-out-Strut. The robotically performed steps are indicated in bold. Note that between each assembly step listed below, a robotic inspection was performed with the on-board camera to collect the data for the SF-GraphSLAM analysis.

Photos of the truss in various stages of the assembly process are shown in Figure 18.

Table 3. Requirements list for the physical truss hardware with green boxes for successfully met criteria and yellow boxes for criteria met after additional improvements.

Requirement	Verification	Result	Comments
Vertical struts rotate to deploy	Visual	✓
Vertical struts sit at 90 degrees	Measurement	✓
Nodes lock vertical struts in place	Testing	-	Additional lockouts added for security
Deployable allows for proper weight loading	Measurement	✓
All deployables attach to their center close-out-squares	Visual	✓
All close-out-struts can be inserted	Visual	✓
Truss is squared off	Measurement	✓
Truss elements sit in standoffs	Visual	✓

showcases the base-level requirements needed to achieve the full BORG assembly demonstration. They were independently evaluated through a series of visual and measured criteria and continually improved upon until they reached allowable tolerances. The only significant requirement that was not fully addressed was confidence in the lockout ability of the node-based one-way bearing mechanism. To provide additional support to the vertical members during deployment, lockout screws were also manually inserted.

There are three different types of AprilTag relationship cases that can be defined with a single deployable mechanism or assembly constraint: (1) deployable mechanism, (2) close-out-strut assembly, and (3) close-out-square assembly. SF-GraphSLAM has these relations embedded within it and tied to the tag map of all the AprilTags of the BORG structure.

The sections below discuss some of the results of the hardware testing, as well as highlighting some major challenges with the physical testing and recommendations for improvements in future testing.

4.3.1. Deployable Relation

To check the deployable relation, a robotic inspection was carried out after step 2 of the assembly process was completed, and deployable 2D was placed on the turntable platform. The camera was mounted on the Stewart platform at the end of LSMS, and a small sweeping motion of LSMS was used to collect the data set. This check both verified the proper deployment and placement of the truss on the turntable. Additionally, the OptiTrack data can be used as another source of information since there is no ground truth for the hardware testing as was available in the simulation. Figure 19 shows a snapshot of the view from the camera during this inspection. Results for a similar deployable checkout are shown in Table 1, but below, this example is used to highlight improvements that could be made in the secondary verification with the global metrology system OptiTrack data.

As shown below in Figure 20, the multiple sources of data from the hardware trials were integrated to be able to feed into the same GraphSLAM architecture used for the simulation. The resulting estimation of the relative positions of the top and bottom tags of the deployable are both quite close to the ideal 0.5 m using the camera data and OptiTrack data individually.

Positional offset transforms between the OptiTrack marker coordinate frames and the AprilTag coordinate frames were calculated based on the arrangement of the OptiTrack reflective markers. Unfortunately, the rotation of the coordinate frame was generated based on the metrology marker’s relation to the global coordinate frame at the time of OptiTrack object creation. This created a non-predictable rotational offset. Therefore, the OptiTrack pose could not be perfectly converted to the frame of the camera’s GraphSLAM and SF-GraphSLAM results because the rotational offset error compounded when being multiplied by the other transforms, causing rotational and positional error shown in Figure 21. For future testing, the metrology marker tags should have the axis manually defined in the OptiTrack software based on the hardware right angles for a predictable coordinate frame. Therefore, direct comparison of the entire pose of the OptiTrack and SLAM estimates is not useful, and the distances of the tag relations in each are compared instead to provide a secondary reference of the state of the hardware during testing.

4.3.2. Close-out-Strut Relation

To test a close-out-strut relationship and look at the state of the truss at the end of layer 1, the next step highlighted is the robotic checkout after step 6. The view from the camera during this test is shown in Figure 22. The pose estimates generated from GraphSLAM and SF-GraphSLAM are shown in

Table 4. Pose estimates of the close-out-strut inspection test.

Tag Number (Location in Image)	SLAM	X_Trans (m)	Y_Trans (m)	Z_Trans (m)	X_Rot (rad)	Y_Rot (rad)	Z_Rot (rad)
Tag 7 (Corner Deployable Bottom)	GraphSLAM	0.0032618	−0.3106846	1.34712022	0.83257264	−0.0464243	−0.5520683
	SF-GraphSLAM	0.0855154	−0.2392015	1.05279175	0.88337071	−0.0159959	−0.3862347
Tag 12 (Insertion Strut)	GraphSLAM	−0.41730999	−0.2491637	1.08578045	0.95755029	0.01569564	−0.2753446
	SF-GraphSLAM	−0.3792887	−0.0721678	1.12116063	0.88338201	−0.0159691	−0.386176
Tag 45 (Bottom Right Turntable)	GraphSLAM	0.28395107	−0.1396594	1.39699767	0.84429191	0.01767223	−0.5360217
	SF-GraphSLAM	0.29479852	−0.4393834	1.04855031	0.88336358	−0.0159858	−0.3862082
Tag 46 (Bottom Left Turntable)	GraphSLAM	−0.43205479	−0.0403225	0.95670014	0.90418669	0.08140275	−0.3670429
	SF-GraphSLAM	−0.50936866	−0.1514242	1.16583825	0.88338808	−0.0159323	−0.3861891

and are plotted in Figure 23.

This case shows a potential source of error in hardware trials in terms of limited data sets. For this example, a short, almost motionless video recording was taken with the camera and from a very oblique angle to the structure and turntable. AprilTag accuracy degrades when at more extreme angles, and the lack of movement prevented either SLAM from having additional camera poses to try to improve their tag pose guesses with. Figure 23 shows that the SF-GraphSLAM is still trying to fit the known relations of the tags.

4.3.3. Close-out-Square Relation

The last unique module is the close-out-square which can be seen being assembled in its vertical configuration after step 10, shown in Figure 24. To be able to view all the AprilTags on this first completed face, the long-reach manipulator moved the robot in a large arc up and down.

4.3.4. Full BORG Truss

Now that the entire structure is completed, a full pose estimate can be generated for the BORG truss structure and compared to the ideal simulation model. Figure 25 shows a snapshot of the fully assembled BORG from the robot’s camera.

A single run of the GraphSLAM and SF-GraphSLAM for this video data set was not performed due to the long duration of processing that increases as the number of camera poses and the number of AprilTags observed increases. In general, the SF-GraphSLAM always took longer to optimize than the GraphSLAM when both started from initial guesses of zero for all state vectors. This is because of the additional optimization required for the semantic s function that generates the guess poses for tags based on other observed tags and their known relations. The runtime of the SF-GraphSLAM approach can be improved by using the known map to create a guess for the AprilTag poses in $x0$ based on the measurements for the observable tags at time $t=0$ and then generating guess poses for all the other expected observable tags for the trial using the known map relations. This initial guess generally allowed the SF-GraphSLAM state vector optimization to have a similar runtime to the GraphSLAM.

In general, due to the error in the physical structure, the SF-GraphSLAM semantic relations were not able to be applied constantly since the measurements did not fall within the bounds of the expected relations, which would trigger the ideal transform optimization in the code. This is due to the fact that if there is actually error, SF-GraphSLAM does not want to override it, but instead if the structure is within the required bounds of the ideal, then those ideal map relation transforms can be used to help eliminate measurement and camera pose error and noise. Since the confidence in this low-fidelity structure is not high, the SF-GraphSLAM and GraphSLAM results are much more comparable than the improvement of SF-GraphSLAM shown in the simulation. Further hardware trials with higher precision structures would allow better comparison and analysis of the SF-GraphSLAM method. Furthermore, further analysis would greatly benefit from reliable global metrology verification data.

5. Conclusions

Getting to a point where extremely large structures can be autonomously robotically assembled in space will take a lot of continued development. This demonstration is one of the first steps the FASER Lab is taking to push the state of the art in this field. The BORG demonstration infrastructure will allow the lab to continue to test increased autonomy capabilities using the 3 × 3 × 3 example structure. This paper highlights the hardware development, test infrastructure, the proof of concept of the operations, and preliminary results from the initial teleoperation testing of the mixed assembly scheme.

The results show the successful creation of an example BORG mixed assembly 3 × 3 × 3 module truss structure. This allowed for beginning to test the SF-GraphSLAM using hardware. The preliminary results are included, and further physical testing with higher fidelity and accuracy truss structure is recommended.

5.1. Future Work

For this testing, only the lifting hook and Stewart platform positioning were utilized since the node and strut grippers manufacturing could not be completed, but the full design of JACS is shown in Figure 26. This restricted the robotic capability for this testing demonstration to only include the deployment of trusses, placement of trusses, and inspection of all modules after the completion of assembly steps.

These interaction features shown allow the JACS unit to load an insertion element into a set of grippers and interface with the existing truss elements to fully insert the close-out structure. This gripping ability is vital to the structural insertion task, as prior to close-out, the structure is improperly supported. By gripping onto the critical nodes, the structure can be stabilized and held in position as insertion tasks take place as seen in Figure 27.

Further testing of robotic insertion tasks for the close-out-struts and close-out-panels is desired. This would verify a fully seated element in the capture window to provide reasonable support in the structure. Due to symmetric geometry in the insertion elements as well as specific axes, there also exists the possibility for an insertion to take place when the element is in an improper rotation. This is less crucial in close-out-struts, but for load paths it is more critical in close-out-panels. Initial testing took place to verify that tags were in a “required” location for correct assembly. However, further steps can be taken to identify the states where fully inserted elements are at the incorrect orientation.

5.2. Consideration for a Future Space Application of This Approach

5.2.1. Advanced Ground Testing

This testing was the first round of ground experimentation and did not attempt to make a space-like environment for the manipulation tasks. Future testing could use testing equipment such as (1) air-bearing tables, (2) six-degrees-of-freedom (DOF) motion platforms, (3) cable offloading systems, (4) neutral buoyancy, (5) Zero-G aircraft, or (6) simulation environments to provide a better analog to the desired space operation environment. Air-bearing tables allow for zero gravity simulation for planar movement by providing an air cushion for objects to glide on a flat surface, with very low friction [22]. In our experiment, the turntable represents an area where a turntable could have provided a much more frictionless alternative. Six-DOF motion platforms, such as Stewart platforms or robotic arms, can be used to mount equipment, sense forces being applied, and move in response to simulate a free-floating object’s response to that force [23]. Cable-driven offloading systems can attach to objects at multiple points and suspend them to simulate zero gravity, using sensor feedback to retract the cables to the required positions based on force input [24]. Neutral buoyancy testing utilizes a large pool to submerge testing equipment to allow for 6-DOF testing with simulated weightlessness [25]. Zero-G aircraft can allow for sub-minute testing with weightlessness achieved at the top of the parabolic flight path but are limited by size constraints, so a smaller scale truss would need to be utilized, and due to the limited amount of weightlessness provided, experiments would have to be restricted to those achievable in that time frame [26]. Additionally, digital simulations could also provide another avenue to test the zero gravity applications of the mixed assembly approach. Any of these options would further increase the validity of these experiments in terms of showing the more realistic zero gravity environment of robotic in-space assembly.

5.2.2. Space Rated Sensors and Truss Materials

The In-Space Servicing, Assembly, and Manufacturing (ISAM) State of Play records the current inspection and metrology equipment in development or previously flown [27]. Any space-rated camera could be used for a future version of this testing. Additionally, there are some environmental aspects such as lighting conditions that would need to be taken into account to collect proper camera data and perform SLAM in-space conditions. Independent light sources can be placed on the robotic end effector to illuminate the workspace to provide constant visibility of truss module fiducials during inspection.

The truss modules used for this experimentation were developed using 3D-printed node and aluminum strut components to allow for rapid hardware iteration and testing. For space versions of this hardware, it would be beneficial to use composites for the truss struts due to their low coefficient of thermal expansion (CTE). The nodes should be created out of a higher-tolerance material such as metal. An example of a more space-ready assembled truss structure in development is the TriTruss by NASA Langley Research Center [28].