Abstracting Data in Distributed Ledger Systems for Higher Level Analytics and Visualizations
Abstract
:1. Introduction
2. Related Work
2.1. Visualization Tools
2.2. Block Explorers
2.3. Abstraction Services
3. Proposed Approach
- Parsing is the process of converting raw low-level data structures into higher level objects. Blockchain data structures are optimized for transaction validations and data retrieval across a distributed network and thus are not best suited for conducting analysis easily. For instance, to implement the proposed transaction-oriented abstraction, the parsing procedure must first collect the transactions from one or several blocks prior to mapping their inputs to previous transaction outputs. In addition, transactions must be assigned with IDs and timestamps along with the associated addresses of creators and beneficiaries to facilitate retrieval procedures [43] different parsing procedure;
- Aggregation refers to the collection and integration of data from multiple sources into a single storage destination. During this process, the different data sources required to infer higher level information are gathered and stored within a common data structure. For example, the proposed transaction-oriented abstraction needs to establish links between transaction inputs and outputs. To derive this mapping, transaction metadata of different blocks is aggregated, and corresponding source and destination addresses are matched;
- Caching is the process of storing data resulting from previous computations so that future requests for that data can be executed faster. Both hardware and software used for caching depend on critical requirements such as the data volume, persistence time, access rate, throughput, and format. In the present scenario, the parsed and aggregated data can be cached in a server hard drive and RAM using a regular or graph database. The latter usually provides a better basis for analyzing relationships between entities [44];
- Pre-processing defines the operation of taking the cached data as input to generate the information requested by the query services. For example, this step is necessary to compute statistical insights on the network state, i.e., number of transactions per day. In addition to cached data, the pre-processing operation can also request data from third-party services. In the case of the proposed account-oriented abstraction, a pre-processing service will access the stored aggregated data to cluster addresses based on various possible heuristics [45]. Entities can then be inferred from the clustered accounts. Address clustering is particularly powerful when combined with labeling, i.e., labeling clusters with real-world entity designations [46]. On a small scale, labels can be determined by users through the query services. However, for large-scale labeling, automated scraping of open-source information or access to a third-party service provider is desirable;
- Query processor refers to the interfaces allowing third-party applications or users to query high-level data through a set of predefined instructions. Queries can initiate reading pieces of information collected or generated by the other abstraction layer services. Through a set of rich queries, this service aims to deliver requested data in a readily consumable format. To build the proposed visualizations, the query services can be implemented using Representational State Transfer (REST) APIs and the JavaScript Object Notation (JSON) file format. The defined set of APIs will allow client applications to remotely execute pre-processing services to submit labels and clustering rules before querying the pre-processed data.
4. Use Case Scenario
4.1. The RegNet Platform
4.2. Architecture
4.3. Consumer-Driven Contract Testing
4.4. Queries
4.5. Network Visualization
4.6. Dashboard Concept
5. Discussion and Outlook
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- Casino, F.; Dasaklis, T.K.; Patsakis, C. A systematic literature review of blockchain-based applications: Current status, classification and open issues. Telemat. Inform. 2019, 36, 55–81. [Google Scholar] [CrossRef]
- Feng, H.; Wang, X.; Duan, Y.; Zhang, J.; Zhang, X. Applying blockchain technology to improve agri-food traceability: A review of development methods, benefits and challenges. J. Clean. Prod. 2020, 260, 121031. [Google Scholar] [CrossRef]
- Alladi, T.; Chamola, V.; Rodrigues, J.J.; Kozlov, S.A. Blockchain in smart grids: A review on different use cases. Sensors 2019, 19, 4862. [Google Scholar] [CrossRef] [Green Version]
- Ali, O.; Ally, M.; Clutterbuck; Dwivedi, Y. The state of play of blockchain technology in the financial services sector: A systematic literature review. Int. J. Inf. Manag. 2020, 54, 102199. [Google Scholar] [CrossRef]
- Kshetri, N. Blockchain’s roles in strengthening cybersecurity and protecting privacy. Telecommun. Policy 2017, 41, 1027–1038. [Google Scholar] [CrossRef] [Green Version]
- Wang, J.; Wang, S.; Guo, J.; Du, Y.; Cheng, S.; Li, X. A summary of research on blockchain in the field of intellectual property. Procedia Comput. Sci. 2019, 147, 191–197. [Google Scholar] [CrossRef]
- Wang, Y.C.; Chen, C.L.; Deng, Y.Y. Authorization mechanism based on blockchain technology for protecting museum-digital property rights. Appl. Sci. 2021, 11, 1085. [Google Scholar] [CrossRef]
- Agbo, C.C.; Mahmoud, Q.H.; Eklund, J.M. Blockchain technology in healthcare: A systematic review. Healthcare 2019, 7, 56. [Google Scholar] [CrossRef] [Green Version]
- Ratta, P.; Kaur, A.; Sharma, S.; Shabaz, M.; Dhiman, G. Application of blockchain and internet of things in healthcare and medical sector: Applications, challenges, and future perspectives. J. Food Qual. 2021, 2021, 7608296. [Google Scholar] [CrossRef]
- Kadam, S. Review of distributed ledgers: The technological advances behind cryptocurrency. In Proceedings of the International Conference Advances in Computer Technology and Management (ICACTM), Pune, India, 23–24 February 2018. [Google Scholar]
- Dinh, T.T.A.; Liu, R.; Zhang, M.; Chen, G.; Ooi, B.C.; Wang, J. Untangling blockchain: A data processing view of blockchain systems. IEEE Trans. Knowl. Data Eng. 2018, 30, 1366–1385. [Google Scholar] [CrossRef]
- Zhu, Y.; Zhang, Z.; Jin, C.; Zhou, A.; Qin, G.; Yang, Y. Towards rich Qery blockchain database. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, Virtual, 19–23 October 2020; pp. 3497–3500. [Google Scholar]
- Przytarski, D.; Stach, C.; Gritti, C.; Mitschang, B. Query Processing in Blockchain Systems: Current State and Future Challenges. Future Internet 2021, 14, 1. [Google Scholar] [CrossRef]
- Hegarty, M. Diagrams in the mind and in the world: Relations between internal and external visualizations. In Proceedings of the International Conference on Theory and Application of Diagrams, Cambridge, UK, 22–24 March 2004; pp. 1–13. [Google Scholar]
- Liu, Z.; Stasko, J. Mental models, visual reasoning and interaction in information visualization: A top-down perspective. IEEE Trans. Vis. Comput. Graph. 2010, 16, 999–1008. [Google Scholar] [PubMed] [Green Version]
- Tovanich, N.; Heulot, N.; Fekete, J.D.; Isenberg, P. Visualization of blockchain data: A systematic review. IEEE Trans. Vis. Comput. Graph. 2019, 27, 3135–3152. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Oscar, N.; Mejía, S.; Metoyer, R.; Hooker, K. Towards personalized visualization: Information granularity, situation, and personality. In Proceedings of the 2017 Conference on Designing Interactive Systems, Edinburgh, UK, 10–14 June 2017; pp. 811–819. [Google Scholar]
- Polyviou, A.; Velanas, P.; Soldatos, J. Blockchain technology: Financial sector applications beyond cryptocurrencies. Multidiscip. Digit. Publ. Inst. Proc. 2019, 28, 7. [Google Scholar]
- Zhou, E.; Sun, H.; Pi, B.; Sun, J.; Yamashita, K.; Nomura, Y. Ledgerdata Refiner: A Powerful Ledger Data Query Platform for Hyperledger Fabric. In Proceedings of the 2019 Sixth International Conference on Internet of Things: Systems, Management and Security (IOTSMS), Granada, Spain, 22–25 October 2019; pp. 433–440. [Google Scholar]
- Treleaven, P.; Sfeir-Tait, S. Future Data-Driven Regulation; Technical report; UCL: London, UK, 2020. [Google Scholar]
- Pithadia, H.J. Algorithmic Regulation using AI and Blockchain Technology. Ph.D. Thesis, UCL (University College London), London, UK, 2021. [Google Scholar]
- Rauchs, M.; Glidden, A.; Gordon, B.; Pieters, G.C.; Recanatini, M.; Rostand, F.; Vagneur, K.; Zhang, B.Z. Distributed Ledger Technology Systems: A Conceptual Framework; Cambridge Center for Alternative Finance, Judge Business School: Cambridge, UK, 2018. [Google Scholar]
- Kuzuno, H.; Karam, C. Blockchain explorer: An analytical process and investigation environment for bitcoin. In Proceedings of the 2017 APWG Symposium on Electronic Crime Research (eCrime), Phoenix, AZ, USA, 25–27 April 2017; pp. 9–16. [Google Scholar]
- Ethviewer. Available online: http://ethviewer.live (accessed on 14 December 2022).
- Yue, X.; Shu, X.; Zhu, X.; Du, X.; Yu, Z.; Papadopoulos, D.; Liu, S. Bitextract: Interactive visualization for extracting bitcoin exchange intelligence. IEEE Trans. Vis. Comput. Graph. 2018, 25, 162–171. [Google Scholar] [CrossRef]
- Daily Blockchain. Available online: https://dailyblockchain.github.io/ (accessed on 14 December 2022).
- Bitforce5. Available online: https://www.bitforce5.com/ (accessed on 14 December 2022).
- Bistarelli, S.; Santini, F. Go with the-bitcoin-flow, with visual analytics. In Proceedings of the 12th International Conference on Availability, Reliability and Security, Reggio Calabria, Italy, 29 August–1 September 2017; pp. 1–6. [Google Scholar]
- Blockchain.com. Available online: https://www.blockchain.com/explorer/ (accessed on 12 December 2022).
- Di Battista, G.; Di Donato, V.; Patrignani, M.; Pizzonia, M.; Roselli, V.; Tamassia, R. Bitconeview: Visualization of flows in the bitcoin transaction graph. In Proceedings of the 2015 IEEE Symposium on Visualization for Cyber Security (VizSec), Chicago, IL, USA, 25–26 October 2015; pp. 1–8. [Google Scholar]
- BitInfoCharts. Available online: https://bitinfocharts.com/bitcoin/explorer/ (accessed on 14 December 2022).
- Bitnodes. Available online: https://bitnodes.io/ (accessed on 14 December 2022).
- Hyperledger Explorer Github. Available online: https://github.com/hyperledger/blockchain-explorer (accessed on 30 November 2022).
- Etherchain. Available online: https://etherchain.org (accessed on 30 November 2022).
- Ethplorer. Available online: https://ethplorer.io (accessed on 30 November 2022).
- Etherscan. Available online: https://etherscan.io/ (accessed on 30 November 2022).
- Blockscout. Available online: https://blockscout.com (accessed on 30 November 2022).
- Hyperledger Explorer Documentation. Available online: https://blockchain-explorer.readthedocs.io/en/main/ (accessed on 14 December 2022).
- Thinkit.co.jp. Available online: https://thinkit.co.jp/article/18190 (accessed on 13 December 2022).
- Alethio. Available online: https://explorer.aleth.io (accessed on 30 November 2022).
- Trihinas, D. Interoperable Data Extraction and Analytics Queries over Blockchains. In Transactions on Large-Scale Data-and Knowledge-Centered Systems XLV; Springer: Berlin/Heidelberg, Germany, 2020; pp. 1–26. [Google Scholar]
- Tal, Y.; Jannis, P.; Brandon, R. The Graph. Available online: https://thegraph.com/ (accessed on 30 November 2022).
- Kalodner, H.; Möser, M.; Lee, K.; Goldfeder, S.; Plattner, M.; Chator, A.; Narayanan, A. Blocksci: Design and applications of a blockchain analysis platform. In Proceedings of the 29th {USENIX} Security Symposium ({USENIX} Security 20), online, 12–14 August 2020; pp. 2721–2738. [Google Scholar]
- Tsoulias, K.; Palaiokrassas, G.; Fragkos, G.; Litke, A.; Varvarigou, T.A. A Graph Model Based Blockchain Implementation for Increasing Performance and Security in Decentralized Ledger Systems. IEEE Access 2020, 8, 130952–130965. [Google Scholar] [CrossRef]
- Fröwis, M.; Gottschalk, T.; Haslhofer, B.; Rückert, C.; Pesch, P. Safeguarding the evidential value of forensic cryptocurrency investigations. Forensic Sci. Int. Digit. Investig. 2020, 33, 200902. [Google Scholar] [CrossRef]
- Harrigan, M.; Fretter, C. The unreasonable effectiveness of address clustering. In Proceedings of the 2016 Intl UIC/ATC/ScalCom/CBDCom/IoP/SmartWorld. IEEE, Toulouse, France, 18–21 July 2016; pp. 368–373. [Google Scholar]
- Yang, Q.; Liu, Y.; Chen, T.; Tong, Y. Federated machine learning: Concept and applications. ACM Trans. Intell. Syst. Technol. (TIST) 2019, 10, 1–19. [Google Scholar] [CrossRef]
- Androulaki, E.; Barger, A.; Bortnikov, V.; Cachin, C.; Christidis, K.; De Caro, A.; Enyeart, D.; Ferris, C.; Laventman, G.; Manevich, Y.; et al. Hyperledger fabric: A distributed operating system for permissioned blockchains. In Proceedings of the Thirteenth EuroSys Conference, Porto, Portugal, 23–26 April 2018; pp. 1–15. [Google Scholar]
- Hyperledger Fabric Github. Available online: https://github.com/hyperledger/fabric (accessed on 30 November 2022).
- Hyperledger Fabric Documentation. Available online: https://hyperledger-fabric.readthedocs.io/en/latest/whatis.html (accessed on 14 December 2022).
- Nasir, Q.; Qasse, I.A.; Abu Talib, M.; Nassif, A.B. Performance analysis of hyperledger fabric platforms. Secur. Commun. Netw. 2018, 2018. [Google Scholar] [CrossRef] [Green Version]
- Valenta, M.; Sandner, P. Comparison of ethereum, hyperledger fabric and corda. Frankf. Sch. Blockchain Cent. 2017, 8, 1–8. [Google Scholar]
- Shih, D.H.; Shih, P.L.; Wu, T.W.; Liang, S.H.; Shih, M.H. An International Federal Hyperledger Fabric Verification Framework for Digital COVID-19 Vaccine Passport. Healthcare 2022, 10, 1950. [Google Scholar] [CrossRef] [PubMed]
- Chen, C.L.; Shang, X.; Tsaur, W.J.; Weng, W.; Deng, Y.Y.; Wu, C.M.; Cui, J. An Anti-Counterfeit and Traceable Management System for Brand Clothing with Hyperledger Fabric Framework. Symmetry 2021, 13, 2048. [Google Scholar] [CrossRef]
- Stamatellis, C.; Papadopoulos, P.; Pitropakis, N.; Katsikas, S.; Buchanan, W.J. A privacy-preserving healthcare framework using hyperledger fabric. Sensors 2020, 20, 6587. [Google Scholar] [CrossRef] [PubMed]
- Díaz-Santiso, J.; Fraga-Lamas, P. E-Voting System Using Hyperledger Fabric Blockchain and Smart Contracts. Eng. Proc. 2021, 7, 11. [Google Scholar]
- Microservices. Available online: https://microservices.io (accessed on 30 November 2022).
- Micronaut. Available online: https://micronaut.io/ (accessed on 30 November 2022).
- Meyer, B. Applying ‘design by contract’. Computer 1992, 25, 40–51. [Google Scholar] [CrossRef] [Green Version]
- Lehvä, J.; Mäkitalo, N.; Mikkonen, T. Consumer-driven contract tests for microservices: A case study. In Proceedings of the International Conference on Product-Focused Software Process Improvement, Barcelona, Spain, 27–29 November 2019; pp. 497–512. [Google Scholar]
- Sotomayor, J.P.; Allala, S.C.; Alt, P.; Phillips, J.; King, T.M.; Clarke, P.J. Comparison of runtime testing tools for microservices. In Proceedings of the 2019 IEEE 43rd Annual Computer Software and Applications Conference (COMPSAC), Milwaukee, WI, USA, 15–19 July 2019; Volume 2, pp. 356–361. [Google Scholar]
- Sotomayor, J.P.; Allala, S.C.; Santiago, D.; King, T.M.; Clarke, P.J. Comparison of open-source runtime testing tools for microservices. Softw. Qual. J. 2022, 1–33. [Google Scholar] [CrossRef]
- Pact. Available online: https://docs.pact.io/ (accessed on 12 December 2022).
- Ma, S.P.; Fan, C.Y.; Chuang, Y.; Lee, W.T.; Lee, S.J.; Hsueh, N.L. Using service dependency graph to analyze and test microservices. In Proceedings of the 2018 IEEE 42nd Annual Computer Software and Applications Conference (COMPSAC), Tokyo, Japan, 23–27 July 2018; Volume 2, pp. 81–86. [Google Scholar]
- Ogma-Linkurious. Available online: https://ogma.linkurious.com/overview (accessed on 30 November 2022).
Categories | Features | Proposed Approach | Hyperledger Explorer [38] | Ledgerdata Refiner [19] | Datachain [41] |
---|---|---|---|---|---|
Architecture | RESTful API | ✓ | |||
Microservice based | ✓ | ||||
Data management | Processed data persistence | ✓ | ✓ | ✓ | |
Ledger parsing & aggregation | ✓ | ✓ | ✓ | ||
Aggregation of external data | ✓ | ||||
Low level queries | Block browsing | ✓ | ✓ | ✓ | ✓ |
Transaction browsing | ✓ | ✓ | ✓ | ✓ | |
Block & transaction search by ID | ✓ | ✓ | ✓ | ||
High level queries | Statistics on transactions | ✓ | ✓ | ✓ | |
Tracking & tracing transactions | ✓ | ||||
Ledger operation chronology | ✓ | ✓ | |||
Network change report | ✓ | ||||
Enabled Visualizations | Transaction flow & volume | ✓ | |||
Organization & node activity | ✓ | ||||
News feed | ✓ |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Vinceslas, L.; Dogan, S.; Sundareshwar, S.; Kondoz, A.M. Abstracting Data in Distributed Ledger Systems for Higher Level Analytics and Visualizations. Future Internet 2023, 15, 33. https://doi.org/10.3390/fi15010033
Vinceslas L, Dogan S, Sundareshwar S, Kondoz AM. Abstracting Data in Distributed Ledger Systems for Higher Level Analytics and Visualizations. Future Internet. 2023; 15(1):33. https://doi.org/10.3390/fi15010033
Chicago/Turabian StyleVinceslas, Leny, Safak Dogan, Srikumar Sundareshwar, and Ahmet M. Kondoz. 2023. "Abstracting Data in Distributed Ledger Systems for Higher Level Analytics and Visualizations" Future Internet 15, no. 1: 33. https://doi.org/10.3390/fi15010033
APA StyleVinceslas, L., Dogan, S., Sundareshwar, S., & Kondoz, A. M. (2023). Abstracting Data in Distributed Ledger Systems for Higher Level Analytics and Visualizations. Future Internet, 15(1), 33. https://doi.org/10.3390/fi15010033