Evaluation of a Smart System for the Optimization of Logistics Performance of a Pruning Biomass Value Chain

Featured Application

This work can be used as benchmark for performance evaluation of tools to improve supply chain management in general and solid biofuel supply chain in particular. The potential application is for integrated goods distribution management that promotes efficient resource utilization and sustainability.

Abstract

The paper presents a report on the performance evaluation of a newly developed smart logistics system (SLS). Field tests were conducted in Spain, Germany, and Sweden. The evaluation focused on the performance of a smart box tool (used to capture information during biomass transport) and a web-based information platform (used to monitor the flow of agricultural pruning from farms to end users and associated information flow). The tests were performed following a product usability testing approach, considering both qualitative and quantitative parameters. The detailed performance evaluation included the following: systematic analysis of 41 recordable parameters (stored in a spreadsheet database), analysis of feedback and problems encountered during the tests, and overall quality analysis applying the product quality model adapted from ISO/IEC FDIS 9126-1 standard. The data recording and storage and the capability to support product traceability and supply chain management were found to be very satisfactory, while assembly of smart box components (mainly the associated cables), data transferring intervals, and manageability could be improved. From the data retrieved during test activities, in more than 95% of the parameters within 41 columns, the expected values were displayed correctly. Some errors were observed, which might have been caused mainly by barriers that could hinder proper data recording and transfer from the smart box to the central database. These problems can be counteracted and the performance of the SLS can be improved so that it can be upgraded to be a marketable tool that can promote sustainable biomass-to-energy value chains.

1. Introduction

The use of pruning biomass for renewable energy production is one of the renewable energy uses being promoted in Europe. This study was part of an EU EuroPruning project, “Development and implementation of a new and non-existent logistics chain for biomass from pruning”, which was aimed at developing improved logistics chains for biomass from agricultural pruning residues [1]. This includes the development of a new decision tool for a pruning biomass trading system and logistics management (see Figure 1 and Figure 2). To fulfill this, a smart logistics system (SLS) was developed as described by Gebresenbet et al. [2]. Computer technology has changed the historical face-to-face communication in trading systems [3], and the SLS enables utilization of recent technology.

Understanding and managing the material and information flow (see Figure 1) enables efficient and effective utilization of resources. The SLS developed for this purpose has been reported in detail in [2]. But there are associated research questions: What is the performance of the SLS tool in real conditions? What is the feedback of end users of the tool?

The objective of this study was to evaluate the performance of the SLS under field conditions. This is important, as the SLS is a newly developed tool and evaluating its performance is important. This report describes the methodology applied and the results obtained from applying the SLS tool in real conditions. This is important for replication of the study and improvement of the SLS tool.

1.1. Pruning Biomass Logistics Chain

Figure 1 and Figure 2 describe activities at different stages of a logistics chain. Pruning fruit trees (removing top or unwanted branches) (see Figure 2a) is done by farmers on an annual or biennial basis to maintain the desired tree form and structure and to increase the productivity of fruit trees. Pruning harvesting is often integrated with chipping or baling (see Figure 2b). The harvested product can be stored in the form of bales or chips (see Figure 2c).

Storage of biomass is related to seasonal variability and logistics operation cost, and storage locations should be efficient to reduce transportation and operation costs. The location of biomass storage can vary depending on the nature of the biomass supply chain [4]: on-field storage, intermediate storage (between the fields and power plants), and storage facility at the biomass power plant. Biomass transportation is an important activity in biomass-to-energy systems. This includes both on-farm transportation and main transportation (from farm to storage or power plant).

1.2. Smart Logistics System and Its Utilization by Different Actors

The SLS has four major components, as indicated in Figure 3 and presented in detail by Gebresenbet et al. [2]. The smart box is a sensor unit for measuring parameters such as relative humidity, temperature, geographic position, and route tracking, and information associated with quick read (QR) codes. The onboard control unit is used for planning transport routes and monitoring recordings by Cargolog. The information platform is used for documentation and data sharing, and to facilitate biomass trading and management of the pruning supply chain and traceability. The central control unit is a point of administration of the biomass trading and logistics system and links the information platform and onboard control unit.

The two components of the SLS (the smart box and information platform) have been integrated so that the smart box is functionally connected with the web address of the information platform [5]. The smart box records and transmits data to the central information platform. The information platform is designed to facilitate interactions among biomass supply chain actors and data collection, as well as the management of the entire logistics of the pruning biomass supply chain (see Figure 2 and

Table 1. Actions to be performed by main actors of the pruning supply chain.

Action	Actor	Description of Actions to Be Performed
Add new user	Administrator	Logistics manager who administers the smart system can add new users to the system. New label codes will be assigned to new actors.
Add new pruning	Farmers	Farmers have to input pruning quality parameters required by end users.
Edit pruning	Farmers	Farmers can update input information such as pruning quality parameters while it is in “pending” status.
Delete pruning	Farmers	Farmers can delete pruning as long as it is still pending and has not been flagged as “ready”.
Make ready	Farmers	Farmers can flag pruning as “ready” so that consumers can view that pruning and order it.
Generate QR code	Farmers	Farmers can generate a quick read (QR) code for “pending”, “ready”, and “ordered” pruning.
Order pruning	Consumers	Consumers order pruning based on quality parameters. Consumers select a transporter from a list registered in the central system.
Add to shipment	Transporters	Transporters select orders to be included in one shipment in order to plan the route between locations of chosen orders.
Plan route	Transporters	Transporters generate driving instructions for source and destination locations for all orders included in one shipment.
Add to pruning	Traders	Traders use the tool to select which orders are part of a new treated pruning offer. Trader can add new pruning quality parameters, e.g., if bales are chipped.

). Table 1 presents how different actors use the platform to interact regarding the flow of materials and related information along the pruning biomass supply chain.

The SLS will be used to improve the performance of the biomass supply chain focusing on pruning biomass. In evaluating the performance of supply chain management, it is important to identify the performance measurement metrics within the context of the supply chain under evaluation [6]. The overall performance of a supply chain depends on the role of each stakeholder in the chain [6,7]. Shashi et al. [6] discussed that stakeholders’ interest, value addition, and partners’ performance play important roles in the overall performance of a supply chain.

The rest of this paper is structured as follows. Section 2 describes the testing of the SLS tool and evaluation methodology. Section 3 presents the performance evaluation results and discussion, while Section 4 presents the conclusion.

2. Testing and Evaluation Methodology

2.1. Usability Testing

Usability testing is an evaluation method that has the largest impact on product [8]. It is one method of evaluating the learning and use of new products. In this study, the usability testing approach was used considering six basic characteristics: system to evaluate, focus, participants, tasks, data, and results [8]. These characteristics were applied as explained in

Table 2. Usability testing characteristics.

Basic Characteristics	Description as Applied in SLS Testing
System to evaluate	Used to evaluate the SLS. Usability testing is an evaluation approach that can be applied to evaluate almost any product or technology; for example, software for database management, network management tools, early-stage prototypes, and related help manuals.
Focus	The focus was on usability of the SLS. The test was intended to validate the first release of the smart system (smart box and information platform) prototype, with less consideration of marketability of the product.
Participants	The intended main participants were (potential) end users of the smart system who were actors of the pruning supply chain (farmers, pruning traders, transporters, power plants). Test participants and administrators interacted during tests. Training and instruction guides were provided before starting. During this testing, research centers such as Leibniz Institute for Agricultural Engineering Potsdam-Bornim (ATB) in Germany, Research Centre for Energy Resources and Consumption (CIRCE) in Spain, and the Swedish University of Agricultural Sciences (SLU) as well as Gruyser (transport dealer) participated actively.
Tasks	The smart system has different functional features. Each participant considers the specific functionality of the system that best fits him/her. Test participants who act as farmers perform tasks that the pruning producers perform, while traders perform what processors and traders of biomass perform using the system. Similarly, transporters and consumers test features of the system that serve the transport company and end consumers of the pruning biomass.
Data	Pruning biomass–related data were recorded, stored in a database, and analyzed. Both problems and positive aspects noticed during tests were analyzed. Data collected with three prototypes were used to evaluate the functionality of similar features of the system but with different smart box prototypes. Accordingly, a data triangulation test approach was used, especially to test data storage and display capability of the central information platform by analyzing data sourced from testing with three different prototypes.
Results	Test results were used to improve the smart system and communicate the research outputs. These analysis results are documented, archived, and used to identify what problems surfaced and how to solve them.

. Both qualitative and quantitative data analyses were used in the evaluation of SLS.

2.2. Metrics and Product Quality Model for Evaluation of SLS

2.2.1. Metrics for Performance Evaluation

The SLS deals with data recording, transfer, and storage in a centralized database through the integrated action of a smart box and a web-based information platform (see Figure 3). The functional features of the smart box tool and information platform were integrated using Cargolog PC software, Cargolog FAT90 V2 (Mobitron, Huskvarna, Sweden). In this report, as part of the evaluation of the smart system, metrics were selected and adapted from international standards for systems and software quality requirements and evaluation (ISO/IEC 25010:2011) as described in [9] and

Table 3. Quality attributes considered for performance evaluation of the SLS.

Attribute	Description
Functionality	How easy it is for the system to integrate its functional units while maintaining the security and accuracy of service provided
Reliability	How well the smart system can provide service with required precision
Usability	How easy it is for the smart system to learn, operate, and analyze the data and make decisions
Efficiency	Amount of resources and time required by the system to perform its intended function
Maintainability	How easy it is for the system to identify and fix an error
Portability	How easy it is to move the smart box from one place to another and to move the application platform from one server to another

.

2.2.2. Product Quality Model

The general product quality model for internal and external quality evaluation of information technology (software product) described in ISO/IEC FDIS 9126-1 (see Figure 4) was adapted for evaluation of the smart system [10]. The evaluation is based on the definitions given in international standards for information technology–based product quality (ISO/IEC FDIS 9126-1) as indicated in Table 3 and Figure 4. The detailed analysis results are presented in Table 9.

2.3. Smart System Testing

2.3.1. Smart Box Installation and Training

Before starting the actual testing in Sweden, Spain, and Germany, training/guidance was provided on proper installation and operation of the smart system. In addition to this training, two user guide documents were prepared and provided to actors involved in testing the tool. The first user guide enables smart box users to install Cargolog PC software on computers for onboard monitoring purposes and to mount the smart box tool (with metal case) appropriately on the truck (see Figure 5). The second guide enables users to use the information platform efficiently.

2.3.2. Smart System Field Testing

In this study, three prototypes were developed. The testing and evaluation activities were done with all the three prototypes during 2015–2016. Although testing of the smart system was done during different stages of the development of the system, this report presents only the testing and evaluation results of the final smart system prototype used by end users during final testing activities. The smart boxes were used at farm-to-storage and storage-to-consumer transport stages.

3. Evaluation Results and Discussion

3.1. Data Retrieved from Central Database of the Smart System

In the spreadsheet-based central database, the data are stored in 41 columns (see

Table 4. Parameters and how they are implemented in the spreadsheet-based database on the platform server under 41 columns.

Column No.	Column Header as Implemented in Database	Description of Parameters under Each Column Header
1	title	Name given to the biomass registered with a unique lot number
2	status	Status of the registered biomass along logistics chain (e.g., delivered or not)
3	providerID	Label code assigned to the provider (i.e., farmer or trader) of the biomass
4	transporterID	Label code assigned to the transport company delivering the biomass
5	consumerID	Label code assigned to the end user that ordered (purchased) the biomass
6	shipmentID	Number to identify the delivery route of the biomass with the indicated lot number
7	Cargolog SerialNo	Identification number of smart box used by the transporter while transporting this specific biomass
8	deliveryCode	Product delivery identification number to monitor the traceability of product movement along the chain
9	latitude	Latitude of the location where the biomass is to be picked up
10	longitude	Longitude of the location where the biomass is to be picked up
11	destinationLongitude	Longitude of the location where the biomass is to be delivered
12	destinationLatitude	Latitude of the location where the biomass is to be delivered
13	lotNumber	LotNumber associated with the specific biomass product under consideration
14	quantity	Quantity of biomass associated with each lot number
15	species	Sources of agricultural pruning where biomass product is produced
16	originClassification	Identification of origin and source according to classification in standard EN-ISO 17225-1:2014
17	tradedForm	Information on traded form of biomass
18	particleSize	Particle size distribution of chips
19	baleDiameter	Diameter of traded bale
20	moistureContent	Moisture content of biomass
21	ashContent	Ash content of biomass
22	densityChips	Density of traded chips
23	densityBale	Density of traded bales
24	calorificValue	Caloric value of traded biomass
25	cropCharacteristics	Additional information regarding biomass production and source
26	pruningDate	Date when farmer pruned fruit or other trees
27	collectionDate	Date when pruning is gathered (on farm) to be transported or processed (e.g., chipping)
28	storageDays	Duration of storage at storage site, in days
29	piled	Information to identify whether biomass is stored as large pile or spread as small heaps
30	chemicallyTreated	Information regarding whether biomass product is chemically treated or not
31	covered	Information indicating whether biomass storage is covered or not
32	moved	Information to identify whether machinery is used to move biomass at storage site
33	contaminated	Information to identify whether biomass has been contaminated with impurities
34	useStorage	Information indicating whether storage is used and if it is located at a different position than the address of farm or trader
35	storageLongitude	Longitude value of storage
36	storageLatitude	Latitude value of storage
37	pickupDate	Date and time when the product is picked up for delivery to intended destination
38	deliveryDate	Date and time when the product is delivered to intended destination
39	createdDate	Date and time indicating when the product is registered on the platform server
40	modifiedDate	Date and time indicating when the status (see column 2) of the product along the delivery process is changed by next actor
41	modifiedByUserID	Label code assigned to the actor who changed the status of product (see column 40)

and Appendix A). Data entered by actors directly on the platform and data transferred from the smart box are stored mainly in the 41 columns, while some additional information can be displayed and visualized on the platform. It is important to test the performance of the smart system by considering how correctly the recorded values and information are displayed in each of the 41 columns. This enables us to understand the performance of the smart system and identify the columns where incorrect information might be displayed. In general, during this test, about 104 records of product lines (with specific lot numbers) were retrieved from the SLS database. This in turn enabled us to identify functional features of the SLS linked to errors in displayed values and to figure out what further improvement would be needed to upgrade the system.

3.2. Data on Product Delivery Information

Using the SLS tool, the system administrator can easily identify products entered into the system, products for which transport routes are planned, and products that have been delivered to end users.

Table 5. Number of products registered in platform during field testing.

Country	Recorded Product with Specific Lot Number (N)	Route Planned for Product Delivery		Product Delivered to Consumer *
		Number	%	Number	%
Sweden	28	21	75	14	50
Spain	65	31	48	NA	NA
Germany	11	4	36	3	27
Total	104	56	54	17	16

* Percentage of delivered products does not necessarily indicate efficiency, because some registered products may not be ordered by traders/consumers; for some products, route may be planned but they may not be delivered to consumers. NA, not available.

indicates that, out of the recorded 104 products, route planning was tested for 54%, while actual product delivery of 16% was confirmed according to the data captured by the smart system.

3.3. Data Recorded and Visualized Directly on Platform

The feedback from users regarding the attractiveness and user-friendliness of the functional features of the smart system is very important to improve the system. The feedback may be on utilization of functional features to perform specific actions and/or visualize the results.

Samples of visualizations of recorded data are presented in Figure 6, Figure 7, Figure 8 and Figure 9, which describe information gathered with smart box 2015085119 (see Figure 6) at a test site in Germany during transport of pruning products registered as Field3, lot number 151008B03R1.

The relative humidity data retrieved from the testing activity in Germany are presented in

Table 6. Example of moisture content data compiled from test in Germany.

Product (Lot Number)	Recorded Moisture Content (%)			Remark
	Storage (start)	Storage (end)	Relative Humidity (RH) Measured by Smart Tool
Field1 (151007B01R1)	31.68	31.68	NA	Data were not transferred properly
Field3 (151008B03R1)	31.68	13.36	78.5–88.9	RH measured during transport
Field7 (151008B05R1)	31.68	13.36	91.8–93.2	RH measured during transport
Field8 (151008B04R1)	31.68	31.68	NA	Route planned, but pruning was not transported

, together with the moisture content of the biomass at the storage site. It must be noted that relative humidity indicates the humidity in the air inside the truck box after loading the woodchips. In the initial moment, the moisture measured was just the air moisture at the site. Once the probe (see Figure 5a) was inserted into the truck box, the air in the internal void volume among the woodchips increased its moisture content until it reached equilibrium with the moisture of the woodchips.

3.4. Information Flow and Product Traceability Performance of the Smart System

The information gathered using the smart system was rich enough to construct an effective traceability map of each registered product along the pruning supply chain [11]. Detailed pruning quality parameters and the product traceability system are provided in [12,13]. The SLS was integrated with the biomass traceability system, with pruning quality and related traceability parameters. The usability testing indicated that all of the traceability-related parameters were implemented on the information platform of the SLS (see Figure 3). The identification of producers (provider ID), transporters (transporter ID), and consumers (consumer ID), and the specific name (title) and production lot number assigned to each registered product are mandatory parameters to trace product movement along the pruning supply chain (see Table 4).

Table 7. Major parameters for traceability information continuity and their recording performance

Column Header (Indicating Parameter) as Implemented in Central Database	Correctly Recorded Values of Registered Products for Which Route Planning Was Done (%)
	Sweden N = 21	Spain N = 31	Germany N = 4	Relevance Level of the Parameter for Traceability Information
title	100	100	100	Mandatory
Status	100	100	100	Supporting information
providerID	100	100	100	Mandatory
transporterID	100	100	100	Mandatory
consumerID	100	100	100	Mandatory
cargologSerialNo	67	0	75	Supporting information
deliveryCode	91	68	100	Very important
lotNumber	100	100	100	Mandatory
pickupDate	67	0	75	Supporting information
deliveryDate	67	0	50	Very important
modifiedByUserID	100	100	100	Supporting information

N, number of routed chains as indicated.

indicates that except for smart box number (Cargolog serial no.), delivery code, pickup date, and delivery date, all important parameters are recorded correctly. The errors in records of Cargolog serial no., pickup date, and delivery date could be mainly due to barriers in data transfer from the smart box to the central database. In the case of Spain, the Cargolog serial no., pickup date, and delivery date were not recorded at all in the central database, indicating that the information was not properly transferred from the smart box tool to the central database.

For illustration purposes, a product from the test in Germany is considered here. For a product registered with the title “Field7” and lot number 151008B05R1, the producer, transporter, and consumer are identified as FDEU020, DDEU012, and CDEU013, respectively (see Appendix A). The corresponding complete product identification code can be constructed as FDEU020-DDEU012-CDEU013. Whenever a product traceability issue is initiated, further detailed traceability-related information can be retrieved from gathered data for each stage of the logistics chain. This can be done by an experienced administrator of the smart system who can interpret all recorded values in each of the 41 columns of the spreadsheet-based central database. The data barriers in delivery date (column 38) and Cargolog serial number (column 7) may hinder the system administrator from confirming the final delivery of the product to the intended consumer. In such cases, the administrator should find additional information by telephone or internet conversation, for instance, by introducing an option (on the platform) for the end consumer to confirm the product delivery. This enables the system administrator to easily construct the product delivery identification code and strengthen the traceability and management of the entire pruning supply [10].

The smart system was developed to effectively gather, monitor, store, and analyze data to improve pruning logistics management (even though the tool could be utilized for any biomass supply chain). As indicated in

Table 8. Analysis results of overall satisfactory level of major functionalities.

Functional Features	Main Actor Responsible for the Activity	Satisfaction Level (Ease of Use of the Tool and Precision of Results) *
		1	2	3	4	5
Data entering (uploading) onto web-based platform	Biomass producer				√
Data recording by smart box	Transporter				√
Data transfer from smart box to central database	Transporter			√
Searching for available product and ordering for purchase	Trader and consumer					√
Whole pruning supply chain management support	System administrator				√
Product quality monitoring and traceability capability	System administrator, consumer				√

* This analysis was done assuming that the system will be used by trained actors and system managers. 1 = poor; 2 = fair; 3 = good; 4 = satisfactory; 5 = very satisfactory.

, this integrated system enables improved performance of pruning supply chain management and product traceability by reducing biomass loss (in terms of quality and quantity), increasing the quality of solid fuel and delivery service, and reducing logistics and transaction costs [2,11,14]. Therefore, evaluation results indicate that the SLS is important tool for management of the biomass value chain and trading system. It enables tracing and tracking of the product, controls product quality, facilitates information flow, reduces management cost, and enables managers to manage the entire supply chain. Table 8 indicates that most of the major functional features performed satisfactorily.

3.5. Evaluation of Smart System Using Product Quality Model

In this section, the product quality model is used to analyze the performance of the smart system. The feedback from the testing was systematically used as input information for the product quality model. This input includes observations of the experts participating in testing the tool (see

Table 9. Evaluation of the SLS using product quality model for internal and external quality assessment (adapted with modification from ISO/IEC FDIS 9126-1).

Attribute	Smart Logistics System Performance Description
Functionality	Suitability: The smart system is designed for management of pruning biomass logistics activities. It is suitable for collecting and managing pruning-related data and provides service for all registered actors of the pruning supply chain. Completeness: The system enables complete data to be obtained regarding the biomass quality and quantity as well as its flow along the supply chain. Accuracy: For more than 95% of data parameters included in the spreadsheet-based database, the system accurately displays the recorded data and information. In some cases, errors were noticed, mainly due to problems encountered during data transfer from the smart box. Interoperability: The smart system effectively integrates different functional units such as GPS and GPRS/GSM devices, temperature and humidity recording tools, and the information platform. The data stored in the central database can be downloaded and easily analyzed using spreadsheets, facilitating further interoperability of the system.
Efficiency	Time saving: The tool enables transporters to plan their best transport routes, reducing driving time and distance. Each producer can use the online platform to announce its products while end users and/or traders can buy the ready products and transport services easily online, where the system is managed by an administrator. Resource utilization: Once the pruning biomass-to-energy value chain is initiated, the smart system facilities the coordinated utilization of available resources owned by actors in the chain. Capacity: The smart system has functional capacity to record, transfer, and store adequate data along the pruning biomass logistics chain from producer to end user. It facilities the traceability of pruning quality and logistics management, leading to economic efficiency.
Usability	Appropriateness: The smart system is appropriate, as it enables recording, documentation, and having adequate data centrally, which in turn facilitates the performance analysis and traceability of pruning biomass. Learnability: The tool has a guiding manual to facilitate training and learning to use the system. Once registered by the administrator, each actor can easily practice and use the information platform. User error protection: Once training is provided, there are fewer user errors when using the smart box. Once the power cable is plugged in, the recording will be triggered (started and ended) by a separate portable scanner used to read QR codes on biomass labels. While using the information platform, each registered actor can record and edit only the data he/she provides but not those of other users. Therefore, user error is minimized and can be easily corrected by the respective responsible actor. Accessibility: The smart system will be accessible to all interested actors involved in the pruning-to-energy value chain. However, each user should be registered by the system administrator first and get a specific identification code (labelling code as described in [2,10]). Understandability: The system can be used effectively if initial training and user guide documents are provided. However, how to interpret some displayed values and how to analyze the gathered data can be difficult for many users and should be handled only by experienced (well-trained) system administrators. Attractiveness: The web-based user interface information platform has integrated up-to-date Google Maps, which increases the aesthetics and attractiveness of the online platform. Operability: Components of the smart box are connected to a recording unit by cables [2] (see Figure 5). It has no plug-in and plug-out system for the cables increasing difficulty during operation, indicating that improvement is required. There should also be power on and off buttons for user-friendly operation.
Reliability	Maturity: The smart system is newly developed and tested for the first time. Further repeated tests and evaluation are recommended to increase its maturity level. Availability: The tool is newly developed and not available on the market. From feedback during testing activities, there is a potential market for the smart system, and the tool could be made available for marketing. Fault tolerance: The test results indicate that all registered pruning products were successfully recorded and stored in the central database with associated product quality characteristics. This indicates that a trader or consumer can confidently order any product registered on the information platform and made ready for sale. Other faults, if any, in relation to real-time data during transport may be tolerable. Recoverability: Once the product is registered in the central platform, much associated information will be generated as the product moves downward along the pruning logistics chain. This increases the recoverability of some missed data, if any.
Security	Confidentiality: Personal information of registered actors is confidential, as it can only be accessed by system administrators (of the information platform). Integrity: The system well integrates data recorded by the smart box and related data provided by pruning biomass producers. Accountability: All registered users are accountable for the information they provide to intended users of the information platform. The system enables pinpointing damages that could happen at any stage along the logistics chain through the pruning traceability system integrated in the smart system. Authenticity: All registered actors have specific codes and access to and recognition of data and information they provided.
Maintainability	Modularity: The smart box components can be disassembled and reassembled by the prototype developer. This facilitates maintenance service. Reusability: The system can be maintained and reused. Maintenance service may be required on average once a year, and some parts of the smart box may need to be replaced. Modifiability: The information platform can be modified based on the interest of users or if additional service is required. Testability: Both the smart box and the information platform are testable. After appropriate maintenance, the system will be well tested before it is used.
Portability	Adaptability: The possibility of adapting to logistics of biomass other than pruning residues, such as forest wood products, was taken into consideration during the development of the smart logistics system. Installability: The smart box has a metal case with a magnetic foot for easy and appropriate mounting on a truck. The metal case is used to protect the smart box from mechanical damage, aggressive gases, liquids, and humidity. Packaging: The smart box needs to be shipped from its manufacturing place to end users. The packaging system for this should be hard enough for protection from damage due to impact. Improved packaging is recommended for this purpose.

).

3.6. Major Recommended Improvements

When the SLS was conceived, it was designed for use in supporting and improving biomass logistics, not only for pruning biomass, but for any type of biomass, such as forestry woodchips and herbaceous agrarian residues. For this purpose, further improvements are suggested so that the SLS can be effectively adapted for biomass-to-energy businesses. The performance test results indicate that the SLS performs data collection and storage in the central database satisfactorily. The stored data and other relevant information enable the tool to support product traceability and supply chain management very effectively. For further improvement and service quality, assembly of the smart box components (mainly the associated cables) and data transfer intervals as well as column arrangement in the spreadsheet format for data storage could be improved more.

In general, each batch of pruning biomass data entered into the online platform directly by actors (farmers, traders, etc.) was found to be well documented in the central database and easily visualized by intended users. However, some errors were noticed regarding data on parameters measured by the smart box (such as relative humidity, temperature, Global Positioning System (GPS) coordinates during biomass transport) and transmitted to the database. These errors may be caused by improper initialization of the smart box, unplugging the power cables before data transmission is complete, or unexpected barriers that could interrupt data transmission from the smart box to the online platform.

The smart system enables users to determine the transport distance and truck speed at the route planning stage. However, the real routes followed during transport are displayed only on maps. The actual driving distance and time (or speed) are not determined and displayed along the route maps. Through further improvement, these values can be either displayed on the maps or documented in the spreadsheet-based central database. To increase effective utilization of the information platform, additional functional options should be created (on the platform interface) for transporters and end consumers where they can confirm product delivery and acceptance, respectively. In addition, a billing system could be included as a component to facilitate trading of biomass products.

4. Conclusions

The testing and evaluation of a smart logistics system was carried out in order to identify functional limitations and important improvements to be implemented. The functional features of the smart box and centralized information platform were considered during testing. The features were designed to enable users to have an effective information acquisition, monitoring, and utilization system while promoting sustainable pruning for the energy value chain. All three smart box prototypes were tested together with the information platform. Based on tests done in Spain, Germany, and Sweden, the performance evaluation of the smart system was carried out by systematically analyzing:

➣

values of parameters recorded and stored in the spreadsheet database with 41 columns (for each pruning product registered with a specific lot number),

➣

feedback and problems encountered during the test, and

➣

selected performance metrics in relation to product quality model adapted from ISO/IEC FDIS 9126-1 standard.

The performance test results indicate that the smart system satisfactorily performed data collection and storage in the central database. The stored data and other relevant information enabled the tool to support product traceability and supply chain management very effectively.

In general, for each batch of pruning biomass (with specific lot number) entering and leaving the biomass supply chain, data entered into the online platform directly by actors (farmers, traders, etc.) were well documented in the central database and easily visualized by intended users. The information platform enabled users to display the locations of pruning farms, biomass storage and processing sites, energy plants, and the planned and/or actual delivery routes according to the needs of different actors, such as raw materials producers, traders, and consumers. However, some errors were noticed regarding the smart box identification numbers (expected to be transferred from the smart box to the central database during product pickup and delivery), product delivery codes (expected to be generated by the smart system), and product pickup and delivery times. These errors might have been caused by improper initialization (while triggering the smart box with a power scan), unplugging the power cables before data transmission was complete, or unexpected barriers that interrupted data transmission from the smart box to the online platform. To counteract these problems and increase information continuity along the pruning supply chain as well as effective utilization of the smart system, additional functional features should be created so that transporters and consumers can confirm product delivery and acceptance, respectively. This option enables retrieval of important information that could be missed due to problems encountered by the smart box during data transfer to the central database and increases system performance so that it can be upgraded to a marketable tool promoting sustainable biomass development for renewable energy generation.