Next Article in Journal
Frequency-Domain Filtered-x LMS Algorithms for Active Noise Control: A Review and New Insights
Previous Article in Journal
Ethylene- and Propylene-Glycol Based Nanofluids: A Litterature Review on Their Thermophysical Properties and Thermal Performances
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Surveying the Environmental and Technical Dimensions of Sustainability in Software Development Companies

by
José Alberto García-Berna
,
Juan Manuel Carrillo de Gea
*,
Begoña Moros
,
José Luis Fernández-Alemán
,
Joaquín Nicolás
and
Ambrosio Toval
Department of Informatics and Systems, University of Murcia, 30100 Murcia, Spain
*
Author to whom correspondence should be addressed.
Appl. Sci. 2018, 8(11), 2312; https://doi.org/10.3390/app8112312
Submission received: 19 September 2018 / Revised: 15 November 2018 / Accepted: 15 November 2018 / Published: 20 November 2018
(This article belongs to the Section Environmental Sciences)

Abstract

:
The growing concern over the state of degradation of the environment has led to a consideration of aspects relating to sustainability in software. Bearing this in mind, we have carried out a survey of practitioners, aiming to gather information about their awareness of sustainable software, not only during the development process, but also throughout the period in which the software is used. Using the data gathered, we studied professionals’ perspectives on sustainability in software development, and were able to get a picture of the current situation of the application of sustainability practices in this sector. We focused on the environmental and technical dimensions of sustainability aimed at extending the longevity of information systems and making them more energy efficient. From the results, we observed that there is a widespread desire to pursue sustainable behavior at work, even though there are not always clear guidelines on how to proceed in this endeavor.

1. Introduction

The 2030 Agenda for Sustainable Development was agreed on at the United Nations (UN) Sustainable Development Summit in September 2015. This new framework for international cooperation is committed to eradicating poverty and to promoting sustainable development between 2015 and 2030, where “nobody should be left behind”. It is composed of 17 Sustainable Development Goals (SDGs), along with 169 targets addressing economic, social, and environmental aspects.
While none of the SDGs is specific to Information and Communication Technologies (ICTs), several targets refer to ICTs being used to pursue the expected objectives. The 2030 Agenda for Sustainable Development recognizes that “the spread of ICT and global interconnectedness has great potential to accelerate human progress, to bridge the digital divide and to develop knowledge societies”. It is with that fact in mind that the International Telecommunication Union, which is one of the associated organizations of the UN, has been actively participating in the discussion on the ICT indicators that will be used to track the SDGs, such as “Individuals using the Internet” or “Individuals owning a mobile phone”.
There is an urgent need to raise awareness about how to innovate ICTs [1] in the effort to find the best way to assist all nations in achieving the SDGs by 2030. Nevertheless, ICTs might also have negative effects on sustainability, such as the generation of electronic waste or excessive consumption of power. In this sense, ICTs play an undeniable role as enablers of more efficient resource usage, and are key tools in education and business operations, which are critical success factors for achieving the UN SDGs [2].
Belkhir and Elmeligi [3] underline the exponentially-growing energy consumption caused by the use of ICT devices in the market today. The generation of this energy that is being demanded is a significant contributing cause of the creation of carbon dioxide, a leading Green House Gas (GHG), as well as producing other global warming pollutants. The authors have found that, if unchecked, the relative contribution of ICT GHG Emissions (GHGE) could grow from roughly 1–1.6% in 2007 to over 14% of the 2016 level of worldwide GHGE by 2040, accounting for more than half of the current relative contribution of the whole transportation sector.
All of this means that managerial policies, employees’ habits, and awareness-raising in the hardware and software industries play an essential role in the sustainability of the sector. We have conducted a survey targeted at software development companies, to assess environmental and technical sustainability concerns as a first step in encouraging more sustainable habits and awareness.
Sustainability should be considered as a new issue in software development enterprises and projects, which needs to be achieved without much delay, according to Tjoa and Tjoa [2]. This same argument was also set out in the national project that supported our research in 2015. In this sense, sustainability should be monitored by all internal audit functions of the organizational entities involved. In this work, of the five dimensions (social, individual, environmental, economic, and technical) stated in the Karlskrona Manifesto [4] and presented at the 2015 Institute of Electrical and Electronics Engineers (IEEE)/Association for Computing Machinery (ACM) 37th IEEE International Conference on Software Engineering, we focus on the environmental and technical dimensions.
Environmental sustainability intersects with software in two domains [5]: “Green in software” (i.e., how can we make software more environmentally sustainable? how can we make the process of building software more environmentally sustainable?); and “Green by software” (i.e., how can we achieve environmental sustainability goals through software?) [6]. Both points of view (i.e., “in” and “by”) are considered in the survey. In addition to the economic, social, and environmental dimensions of sustainability, the human (or individual) and technical dimensions were described in literature [7]. The technical dimension is specifically focused on the longevity of software systems (e.g., maintainability, evolution), and is also addressed in the survey.
Some research on the contribution of the software development process and the impact of the execution of software on sustainability has been carried out so far. The GreenSoft Model provides support to stakeholders regarding all the stages of the software life cycle, with a focus on sustainability [8]. Moraga et al. [9] proposed a code of ethics for computer science practitioners that explicitly consider green elements of concern to evaluate the greenability of software. Although the contribution of software to sustainability may be relatively low in a single device, the inherent reproducibility of the software leads to massive energy savings and a large impact on sustainability if an improvement in the use of resources is achieved [10]. This would be an important contribution considering that the global ICT sector produced an energy footprint of 805 TWh, corresponding to 3.6% of the total in 2015 [11]. In addition, the falling costs of clean energy technologies [12] can help reduce the global ICT carbon footprint.
The rest of this work is structured as follows: Section 2 sets out the materials and methods used in this study (i.e., questionnaire, instrumentation, research approach); Section 3 presents the data gathered and the discussion of the results; and, lastly, Section 4 includes our main conclusions and future work.

2. Materials and Methods

The methodology of this work consists of the development and execution of a survey, a validation of this survey, carried out by experts in software development, the steps followed to collect the target companies, and the procedure used to encourage participation. Figure 1 presents a summary of the methodology that allowed us to create and run the survey.
The questionnaire includes demographic questions (e.g., age, genre, level of studies, role, approximate time since being hired in their current company, number of employees in the company). There are also questions regarding practices that could help companies to be more sustainable, along with ones that address the sustainability of the software development process currently applied by the practitioners in their work. The tool used to carry out the survey was Limesurvey; the full survey took approximately 20 min to complete.
Given the economic growth and relevance of the software sector in Spain, particularly in the Autonomous Community of the Region of Murcia (Comunidad Autónoma de la Región de Murcia, CARM), we carried out the survey in local software development companies. Spain is a developed economy in the European Union (EU). In 2017, its Gross Domestic Product (GDP) placed it in fifth place, just behind Germany, the United Kingdom, France, and Italy (http://ec.europa.eu/eurostat/tgm/table.do?tab=table&init=1&language=en&pcode=tec00001&plugin=1). In the first semester of 2018, the regional GDP in the CARM grew by 2.7% (www.airef.es) with respect to the previous year, and it is expected to increase by 3.2% by the end of 2018 (www.ceprede.es). This rate has been similar to that seen in the Spanish economy (2.8%) and is higher than the rises witnessed in the euro area (2.5%), surpassing even the most relevant economies within the area, such as Germany (2.5%) or France (2.1%) (https://www.imf.org/external/pubs/ft/weo/2018/01/weodata/index.aspx).
The sample under study was analyzed to validate its representativeness. From around 190 software development companies that we were interested to survey, we received responses from 35 firms. This represents the 18.5% of the target companies. In addition, none of the participants had a job in two or more companies, so no duplicate responses were gathered. As shown in Section 3, employees of companies of all sizes participated. For example, Capgemini is one of the world’s largest IT consulting, outsourcing, and professional services companies; likewise, Everis is a Spanish multinational consulting company that covers the telecommunications, financial entities, industry, utilities and energy, banking, insurance, public administrations, and health sectors. Different roles are also included in the sample, ranging from technical to management positions.
The software development sector in the CARM is growing and highly dynamic. The National Statistics Institute of Spain states that it has a total of 1635 companies; the number of contracts in the ICT sector was around 4900 in 2016, and this amount is expected to reach 5050 in 2020 (https://www.murcia.com/region/noticias/2017/01/24-la-region-cuenta-con-unos-4900-puestos-de-trabajo-y-mas-de-1630-empresas-en-el-sector-tic.asp). The turnover of the ICT sector in Spain was Euro 88,334 million in 2016 [13], accounting for 7.9% of the total GDP (EUR 1,118,522 million (https://www.ine.es/en/prensa/cna_2016_1_en.pdf)). According to the National Observatory of Telecommunications and Information Society [13], in 2016 there were 2574 software development companies and 2425 software consulting companies in Spain. The turnover of these companies was EUR 5458 million and EUR 11,281 million, respectively.

2.1. Questionnaire Development

The survey was designed in weekly meetings, and is based on a review of literature, experience, and expertise of the Software Engineering Research Group (GIIS) at the University of Murcia (UMU). We used a 5-point Likert scale, from 1, which means total disagreement, to 5, which means total agreement. The aim was twofold: on the one hand, we investigated the views of the practitioners about the potential effect of certain current issues in software development with respect to sustainability; on the other hand, we also asked the practitioners about the sustainability of the software development process that they apply at work. From the conceptual point of view, therefore, the questionnaire is divided into two parts.
We analyzed the state-of-the-art of sustainability in software; the practices were selected according to the findings of other authors (see Table 1). “Requirements reuse”, “code reuse”, “general sustainability policies (how the company operates)”, “specific sustainability requirements”, “principles or requirements for usability and user experience in the development of products”, “principles or requirements for accessibility in the development of products”, “principles or requirements of internationalization in the development of products”, and “Consideration of sustainability when software systems and services are purchased or outsourced” are practices intended to produce an impact on sustainability during the process of software development [14]. “Agile and lean methodologies” [15], “software product lines” [16], “testing and continuous integration” [17], and “standardization of practices, tools, technical solutions, etc.” [18] are methodologies and techniques in which analyses and estimations of the power required to produce software are studied. The “certification in some sustainability standard (for example, International Organization for Standardization-ISO 14001 and/or Eco-Management and Audit Scheme-EMAS)” was evaluated in Reference [19], revealing that certified organizations carry out energy management practices related to the approach of the paper. “Cloud computing and virtualization services” and the “use of “green” data center infrastructures (using renewable energies)” play an important role for the sustainability of the software development process. In this sense, elements of clouds which contribute to the total energy consumption are discussed in Garg and Buyya [20], and a model for minimizing data centers’ energy expenses and maximizing the service is proposed, taking into account service-level agreements, the availability of local renewable power generation, and data centers’ workloads [21]. “Big data and data mining” was incorporated because it is widely recognized as being one of the most powerful drivers to promote productivity, improve efficiency, and support innovation, and because of the analysis that is made concerning green metrics of big data in Wu et al. [22]. Another practice indicated in the survey was the use of the “Internet of things (IoT)” during the software development process. IoT serves people in a collaborative manner, enhancing connectivity all over the world. Taking into account the possibility of using the green IoT, a contribution to sustainability can be achieved in software development process [23]. “Virtual reality and augmented reality” is another practice that was included in the survey. In Santos and Moura [24], a new software development paradigm focused on people, information, and an environment with augmented reality support is proposed for sustainability in the software. The aforementioned papers illustrate that the practices studied relate to the environmental and technical dimensions of sustainability. We asked practitioners if they applied those practices at work, and if they believed that these alternatives could help companies to be more sustainable.
In the second part of the survey, we also included a subset of the statements proposed by Manotas et al. [14] (see Table 2). We included these statements because they are closely linked to sustainability in software. Moreover, we completed them by adapting some of the technical habits that appear in Manotas et al. [14], in an effort to know more about the perceptions of professionals. These statements were adapted by the GIIS considering different habits to provide sustainability in the software development process. Moreover, the statements in this part are interrelated; we proceed with a classification considering the following groups: (1) Requirements about energy usage, which comprise statements from 1 to 2 in Table 2; (2) Requirement-related statements, expression number 3; (3) Design-related statements, declarations 4 and 5; (4) Construction-related statements, 6 to 9 statements; (5) Finding and fixing issue-related statements, from 10 to 12 in Table 2; and (6) Maintenance-related statements, from 13 to 16.

2.2. Questionnaire Validation

At the beginning of the study, we held three meetings with stakeholders from two institutions related to environmental issues: the General Environmental Administration of the CARM, and Reinicia, a non-profit organization specializing in the life-cycle management of electrical and electronic devices for compliance with environment regulations. The roles of the interviewees were general manager, environment technician, and institutional relations representative. Understanding their viewpoint helped us to structure and articulate the survey better, as well as to avoid possible gaps and misconceptions.
Once an initial version of the survey had been prepared, it was piloted among 20 software development practitioners, in an endeavor to gather external feedback that would improve the questionnaire before its delivery. The experts were provided with a validation questionnaire, divided into four parts: (1) presentation letter (i.e., clarity, suitability, length, and quality of the presentation letter sent to the potential participants in the survey); (2) questionnaire instructions (i.e., clarity, suitability, quantity, and quality); (3) questionnaire items (i.e., logical order of the questions, clarity of the writing, suitability of the answer options, number of questions, adaptation to the recipients, and efficiency in providing the data required); and (4) a general evaluation of the survey (i.e., validity of the survey content). Appraisers rated items on a 4-point Likert scale from 1 (bad) to 4 (excellent). In addition, there were five open questions to make it easy to propose the changes related to each part of the questionnaire, and to make any other remarks.
The feedback obtained was generally positive, and it was only necessary to make small changes. Some recommendations related to the format of the survey, and not to the content itself. Throughout the survey, there were also comments asking for more direct statements and questions, or for clearer explanations.

2.3. Compilation of Participants

To compile a list of the target companies that would be sent the survey, we contacted two public institutions of the CARM: the Development Institute of the Region of Murcia (INFO), and the Chamber of Commerce of Murcia. Some of INFO’s missions include the offer of integral support to entrepreneurs, encouraging business innovation, improving logistics processes, promoting foreign trade, financing companies, promoting investments, and managing European projects. The Chamber offers a series of services to help improve the activity of CARM companies and increase their competitiveness abroad; it also provides information and training, and gives economic, legal, and industrial advice. Lists of ICT companies and, specifically for this study, software development companies, were obtained through both institutions.
Once we had collected the contact information of the target companies, we used Mailchimp to automate and manage the participation requests. This included the creation and submission of invitation emails, and the management of the companies. We created an email template with a general description of the main goals of the study, emphasizing the importance of the employees’ participation, and personalized the message to each recipient company. Those receiving the email were asked to distribute the information among the employees of the company, to boost participation.
The invitation emails directed prospective participants to a webpage that included information about the study, along with a registration form to gather basic contact details about the practitioners (https://giis.inf.um.es:8443/formulario). The data requested for the registration were a name or nickname (not relevant), an email address, and the name of their company. We used these data to generate unique tokens in Limesurvey (i.e., individual participation codes) and to track the survey completion status. Once the form had been submitted, an access link to the survey was immediately provided and sent automatically to the participant’s email address.

2.4. Participation Encouragement

It has been proven that involving participants in the completion of surveys is a difficult task to perform [25]. We followed the recommendations by Escalona et al. [26] to promote participation in the survey. We made our study official with the cooperation of two public organizations from the government of the CARM: The General Environmental Administration, which belongs to the Ministry of Tourism, Culture and Environment, and the General Administration of Information Technology, Heritage and Telecommunications, which belongs to the Ministry of Finance and Public Administration. Support letters for our project were signed by the people at the top of the organization chart of both institutions, namely the Minister and the General Director. Furthermore, we reached an agreement to provide the participants in the survey with an official diploma for participation, signed by the major representatives of the aforementioned institutions. We also obtained a support letter for our study from the President of the UMU and the Vice-president of Economy, Sustainability and Health Sciences of the UMU.
As a general strategy, a reminder email was sent 20 days after the invitation to all prospective participants that had not completed the survey. Since the study’s success relied on the involvement of practitioners, an appropriate dissemination of the survey was critical. We therefore established direct and closer communication with the heads of some of the software development companies, aiming to encourage them (and their subordinates and co-workers) to participate.

3. Results and Discussion

A total of 94 practitioners completed the survey. In Table 3, we display the demographical data. Numbers in bold represent the highest amount of each demographic data field. The results showed that most of the participants (86.17%) were male. Moreover, a large portion of them (40.43%) were between 35 and 44 years old.
Most of the participants had completed higher education, with a much smaller number of cases having completed secondary or vocational education (see Table 3). As shown in Table 3, the most frequent role of the participants in the survey was technical. A relatively small percentage of the respondents (10.64%) were in the category “Other”. This category encompasses participants that play more than one of the base roles (e.g., “Project manager” and “Technician”, “Management” and “Administrative”, “Management” and “Technician”), as well as participants that indicated roles different from the base roles provided in the questionnaire (i.e., “Marketing and sales coordinator”, “Team coordinator”).
The experience of the participants in their current companies was quite varied, as was the size of the companies where the participants in the survey worked (see Table 3). All categories of enterprises were represented; this includes small- and medium-sized enterprises with fewer than 250 employees (i.e., micro-, small-, medium-sized enterprises with fewer than 10 employees, 10 to 49 employees, and 50 to 249 employees, respectively) and large enterprises (250 or more employees).

3.1. Best Practices Evaluation for Sustainability

We collected data on a set of best practices for sustainability. In Figure 2, we compare the percentage of application of these practices, along with the percentage of agreement on suitability for sustainability.
The figures reveal that “Reuse of code” (92.55%), “Reuse of software requirements and specifications” (72.34%), and “Testing and continuous integration” (67.02%) were the most common practices that respondents applied. Moreover, reuse practices were one of the most convincing alternatives that produce the greatest impact on sustainability, with over 80% agreement. This could be explained by the availability of know-how in companies, as well as by the quick access to it. This also leads to a speeding-up of the software development processes, making it more possible to save company resources and costs. “Testing and continuous integration” produces a conviction about sustainability that is slightly different, with between 60% and 80% agreement. This difference could be related to the inherent need for software development iterations, and to the uncertain results produced in each iteration.
Apart from reuse practices and continuous integration, “Standardization of practices, tools, technical solutions, etc.”, “General sustainability policies”, “Specific sustainability requirements” and “Lean and agile methodologies” were perceived as the most effective ways of reaching sustainability. There was a consensus about this among respondents, with over 80% agreement. In addition, these practices provide clear steps on how to proceed when developing software, increasing the sustainability expertise in companies.
“Standardization of practices, tools, technical solutions, etc.” (56.38%) and “Lean and agile methodologies” (57.45%) also had an acceptable percentage of usage. On the other hand, “General sustainability policies” (25.53%) and “Specific sustainability requirements” (14.89%) are not commonly applied at work. This low percentage can be explained by the lack of knowledge and literature/sources, which leads to little time being devoted to sustainability. Consequently, there seems to be a need to promote these practices so that professionals apply them with the same frequency as other practices.
“Cloud computing and virtualization services” (59.57%) and “Usability and user experience requirements or principles in the development of products” (56.38%) are widespread practices in software development companies. Moreover, there is a high level of agreement (between 60% and 80%) among practitioners that these practices help towards sustainability. The existence of certifications specifically designed for data centers, such as the Certified Energy Efficient Data Center Award (CEEDA), may explain the trust in one of these practices. The existence of companies certified in sustainability increases confidence on the part of customers when outsourcing of services occurs. On the other hand, usability and user experience requirements are widely studied, and there are many catalogs available in the literature.
Other practices reached a good level of conviction among the practitioners (also between 60% and 80%). They were “Use of “green” datacenter infrastructures (that use renewable energy)”, “Taking sustainability into consideration when software systems and services are purchased or outsourced”, “Accessibility requirements or principles in the development of products”, “Internationalization requirements or principles in the development of products”, and “Certification in any sustainability regulation (for example, ISO 14001, EMAS...)”. However, not all of these are widely used. The most commonly applied at work are “Accessibility requirements or principles in the development of products” (40.43%) and “Internationalization requirements or principles in the development of products” (40.43%). This fact might be explained by the existence of catalogs that aim to simplify the reuse of requirements, such as the Simple Reuse of Requirements catalog (SIREN) proposed by the GIIS, which saves time in companies and has a beneficial effect on sustainability. Moreover, the need to internationalize to be able to sell abroad is important for companies, especially for technology companies, if they are to survive.
Certifications (11.7%), consider sustainability when purchasing or outsourcing (4.26%), and the usage of sustainable infrastructures (4.26%) were practices that were carried out less in the companies. This reflects the fact that even when there are good intentions to apply best practices for sustainability (with an agreement among practitioners of between 60% and 80%) there seem to be difficulties in putting these alternatives into practice. The reason may be that the costs of certification could very well hold the companies back from implementing the practices, because there are not immediate benefits. The lack of companies certified in regulations for sustainability leads to the impossibility of considering sustainability when purchasing or outsourcing products, or of choosing to use green datacenter infrastructures. There should therefore be a promotion of certification among the companies in the quest to enhance sustainability.
Finally, the group of practices with the least conviction about sustainability comprised “Software product lines” (52.13%), “Big data and data mining” (46.81%), “Internet of things (IoT)” (45.74%), and “Virtual and augmented reality” (28.72%). The level of use of these practices was even lower: “Software product lines” (21.28%), “Big data and data mining” (17.02%), “Internet of things (IoT)” (17.02%), and “Virtual and augmented reality” (3.19%). Practices such as software product lines, big data mining, and the use of IoT have an acceptable reception among professionals, as these technologies are proving productive for client companies, making it possible to produce more specific goods, in less time. However, other technologies such as virtual reality (VR) and augmented reality (AR) create more doubts regarding sustainability. Even though these technologies have traced out a certain path in the market, they are still in their infancy, and there is a lot of work to be done [27].
There was generally a good level of willingness to engage in sustainable behavior at work on the part of the practitioners. This may be observed in the data set out above. The percentage of the perceptions always tends to be higher than the actual usage of the practices. For this reason, practitioners need to be provided with ways of proceeding in a sustainable manner at work.

3.2. Perspectives in Sustainable Practices

In the survey, we gathered specific details on the sustainability of the software development process followed by the respondents. As shown in Figure 3, most participants answered that “I could learn how to improve the use of energy” (78.72% of respondents “Agree” or “Strongly agree”) and “I want to learn about problems with the energy use of my applications” (76.6%). However, only 21.28% of the respondents affirmed that “I learn about problems with the use of energy of my applications”.
Only 20% to 30% of the respondents affirmed that “The applications that I develop have energy use requirements”, “I am willing to sacrifice performance, usability, etc., in exchange for having a lower energy use”, “I take energy use issues into account in design”, “When the use of energy is being evaluated, I take into account the different factors involved (use scenarios, environment, hardware, etc.)”, “Energy use issues have an influence on the way I write code”, “I know the use of energy of my code”, and “When I modify the code, I make changes that I think will improve the use of energy”. It seems that, despite the reported interest in training and the belief in its success, not much attention is currently being paid to sustainability in the development of applications.
The situation is even worse when it comes to documenting the code. The item “I document the code with the changes that are made to improve the use of energy” was supported by less than 20% of the participants in the survey. This fact reveals that documenting the code with a sustainable aim could be affected by a lack of knowledge. The documentation for an acceptable usage of APIs [28] could help achieve a better understanding of how to document the code as regards sustainability. As a matter of fact, writing comments related to sustainability in the code can help to raise awareness among other developers. The same situation occurs when investigating the use of energy from the applications. The item “I find out how the changes I make have an impact on the use of energy” received low support from professionals (less than 20%). In this sense, researchers should strive to let workers know what the main aspects that should be considered in the use of energy in the applications are. The same thing happens when discussing the sustainability of the applications among co-workers. The statement “During code reviews or other discussions, the use of energy is mentioned” found little support. This means that the more awareness there is among professionals, and the more tools there are to assess the energy consumption of software, the more frequent discussions on this issue should be.
The participants believed that technical issues with the use of energy on the part of applications are complex (“Problems with the use of energy are more difficult to discover, diagnose and/or fix than other performance problems”; 56.38% of respondents “Agree” or “Strongly agree”), and applications play an important role in achieving sustainability (“The good use of energy should be the responsibility of the applications”, 52.13%). Nevertheless, the participants in the survey did not believe that “Problems with the use of energy occur more frequently than other performance problems” (12.77%). The responses to these statements reveal that sustainability practices such as usability, quality, and security in software are not trivial, and should be considered throughout the software development processes.

3.3. Statistical Analysis

With the purpose of further analyzing the practices along the demographic data, a statistical analysis was performed. Two independent variables were defined: (i) profile of the respondent (technical versus non-technical); and (ii) size of the company (medium/large versus micro/small). Table 4 shows the significance of the Pearson’s chi-squared statistical test, and the odds ratio (OR) statistic and 95% confidence interval for each combination of independent and dependent variables. Numbers in bold are used to highlight statistically significant outcomes. Column “Set” refers to the specific set of items: sets number 1 and 2 are shown in Table 1 (18 questions); and set number 3 is presented in Table 2 (16 questions). The difference between sets 1 and 2 is that the former refers to whether the respondents apply the practices at work, whereas the latter is focused on whether the respondents believe that these practices can help companies be more sustainable. In addition, the answers in sets 2 and 3 are not dichotomic, so they had to be binarized before performing this statistical analysis. “Strongly disagree”, “Disagree”, and “Neither agree nor disagree” were thus mapped to “No”, and “Agree” and “Strongly agree” were matched to “Yes”.
Participants with a technical role were less inclined (OR = 0.398, 95% CI 0.171–0.924) to apply principles or requirements for usability and user experience in the development of products than those with a non-technical role. The application of principles or requirements for accessibility in the development of products also depended on whether those who did so were technical staff (OR = 0.324, 95% CI 0.138–0.763) in comparison with non-technical staff. The professional category “technical” reduced the probability (OR = 0.357, 95% CI 0.152–0.839) of applying agile and lean methodologies in comparison with managers, project managers, and systems managers. These results are coherent with the idea that the latter are usually more concerned about software requirements and development methodologies than the former. Likewise, participants with a technical role were less inclined to apply cloud computing and virtualization services (OR = 0.419, 95% CI 0.179–0.984) and big data and data mining (OR = 0.089, 95% CI 0.019–0.421) in their companies than those with a non-technical role. Again, this could be explained by nature of the tasks assigned to each type of employee in the company, since non-technical roles are those who make the decisions about the application of those practices.
Regarding the size of the company, medium and large companies (MLCs) were less inclined to apply requirements reuse (OR = 0.328, 95% CI 0.127–0.843) and specific sustainability requirements (OR = 0.159, 95% CI 0.033–0.754) than micro and small companies (MSCs). This might seem surprising, since one could expect larger and more resourceful companies to excel in this regard. However, MSCs in some cases prove to be less corseted than MLCs, and to be more dynamic and innovative. On the contrary, MLCs increased the probability of applying testing and continuous integration (OR = 2.866, 95% CI 1.142–7.193) compared to MSCs. The implementation of testing and continuous integration, and other DevOps practices, is often based on full-scale adoption; this seems to diverge from the philosophy of smaller companies, even though small teams can reap great benefits in adopting these practices. For example, large organizations usually have applications that are difficult to move to continuous delivery, yet they cannot be ignored in the process of continuous delivery adoption because many of them are very important for the success of the company. Employees of MLCs were also less inclined to apply the Internet of Things (OR = 0.219, 95% CI 0.058–0.830) than those of MSCs. This could be due to the same reasons that were argued in relation to the requirements reuse and specific sustainability requirements.
Moving on to the perceived usefulness for sustainability of the previous practices, no significant differences were found across either the role of the participants or the size of the companies. There seems to be a consensus in the opinions expressed by the participants, regardless of their role or the size of their company. In other words, as shown in Section 3.1, there are practices that employees of software development companies believe could help companies to be sustainable and other practices in which they seem to have fewer expectations.
Regarding the questions on the sustainability of the software development process that the participants apply in their companies, we only found significant differences in two items. On the one hand, participants with a technical role were less inclined (OR = 0.398, 0.171–0.924) to believe that problems with the use of energy were more difficult to discover, diagnose, and/or fix than other performance problems in comparison with participants with a non-technical role. On the other hand, the fact that participants took into account the different factors involved (use scenarios, environment, hardware, etc.) when the use of energy was being evaluated depended on whether those who did so were technical staff (OR = 0.385, 95% CI 0.148–1.000) or non-technical staff.

4. Conclusions

A representative sample of software practitioners filled out our survey questionnaire concerning sustainability in software development companies. According to the feedback gathered, there are some practices that can be applied to help companies to be more sustainable. These alternatives could be implemented to make it easier for software development companies to transform into more environmentally-friendly organizations. In addition, we have identified gaps in the software development process that practitioners apply in their companies, especially in relation to the use of the energy of the applications. Practitioners are, however, willing to learn about the use of the energy of their applications. Clear guidelines and protocols need to be developed in software development companies, based on best practices in sustainability, if these organizations are to become greener.
Certifications for sustainability (e.g., ISO 14000 and EMAS) could be another point to consider in seeking more sustainable software businesses. In this regard, it is not only the heads of the companies that are concerned; governments can also play a fundamental role. For instance, when offering funding (see Figure 4), regional institutions such as the INFO look at whether the companies applying for funds possess environmental certifications such as ISO 14001 and EMAS. On the other hand, obtaining sustainability certifications can become a bulky process for the heads of the companies. They may count on specialized consultants or organizations that can provide feedback and support towards this aim. There are also methodologies available in literature that facilitate this objective [29]. Furthermore, sustainability certifications usually target issues (e.g., waste recycling and management, water and energy consumption, paper saving, etc.) that are not close to the particularities found in software development companies. This can also pose a challenge during the certification process.
From the social point of view, our survey notes that the participation of women in software development companies remains low. The long-term sustainability of these companies partly depends on their ability to attract a specialized workforce. It is thus advisable to make more efforts to promote STEM careers among young people, and it is especially recommendable to take the opportunity to involve more women in technology. This would allow women to acquire the skills necessary to access many green jobs and contribute to innovation and development of technology [30].
The fulfillment of the SDGs in the 2030 Agenda for Sustainable Development plays an important role in achieving sustainability objectives, also in technology. They can serve as a general framework that governments and public institutions can follow to carry out their specific sustainable actions. Prescriptions to policy makers were established in Japan with the purpose of effectively implementing the SDGs [31].
In the Region of Murcia, we are seeking to emphasize the relevance of the public sector in leveraging sustainability in software development companies. The Regional Administration and, in particular, the General Administration of Information Technology realize the need to make more efforts in relation to sustainability. In the near future, they may require compliance with environmental regulations by software development companies that apply for contracts and projects in the public sector. We are also planning to undertake actions to raise awareness of sustainability and to study mechanisms that would make people more conscious of their use of resources. We are currently working on energy-saving actions that could be adopted at the university to leverage behavior change among students. These initiatives could reduce energy waste and minimize harm to the environment. We will focus on the main factors that strengthen sustainable behavior and will seek to apply those factors to the professional context.
More efforts need to be devoted to studies on this subject, given the current environmental situation, as well as the relevance of software nowadays. Carrying out sustainability surveys will broaden awareness, allowing for a continued evaluation of the perceptions and habits of professionals about the sustainability of software.

Author Contributions

Conceptualization, J.L.F.-A., J.N. and A.T.; Methodology, J.M.C.d.G., J.L.F.-A. and J.N.; Validation, B.M., J.L.F.-A., J.N. and A.T.; Formal analysis, J.A.G.B., J.M.C.d.G. and J.L.F.-A.; Investigation, J.A.G.B. and J.M.C.d.G.; Resources, J.A.G.B. and J.M.C.d.G.; Data curation, J.A.G.B. and J.M.C.d.G.; Writing—original draft preparation, J.A.G.B., J.M.C.d.G. and B.M.; Writing—review and editing, J.A.G.B., J.M.C.d.G., B.M., J.L.F.-A., J.N. and A.T.; Visualization, J.A.G.B. and J.M.C.d.G.; Supervision, B.M., J.L.F.-A., J.N. and A.T.; Project administration, J.L.F.-A., J.N. and A.T.; Funding acquisition, J.L.F.-A., J.N. and A.T.

Funding

This research is part of the GINSENG-UMU project (TIN2015-70259-C2-2-R) supported by the Spanish Ministry of Economy and Competitiveness and the European Fund for Regional Development (ERDF).

Acknowledgments

The authors would like to express their gratitude to all the people and organizations involved in this study, in particular the survey participants, the European Business Innovation Centre of Murcia (CEEIM), the Technological Centre of Information and Communication Technologies of the Region of Murcia (CENTIC), and the Murcia Science Park for their cooperation in this study.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Wu, J.; Guo, S.; Huang, H.; Liu, W.; Xiang, Y. Information and Communications Technologies for Sustainable Development Goals: State-of-the-Art, Needs and Perspectives. IEEE Commun. Surv. Tutor. 2018, 20, 2389–2406. [Google Scholar] [CrossRef] [Green Version]
  2. Tjoa, A.M.; Tjoa, S. The Role of ICT to Achieve the UN Sustainable Development Goals (SDG). In Proceedings of the IFIP (International Federation for Information Processing) World Information Technology Forum (WITFOR), San José, Costa Rica, 12–14 September 2016; pp. 3–13. [Google Scholar]
  3. Belkhir, L.; Elmeligi, A. Assessing ICT global emissions footprint: Trends to 2040 & recommendations. J. Clean. Prod. 2018, 177, 448–463. [Google Scholar]
  4. Becker, C.; Chitchyan, R.; Duboc, L.; Easterbrook, S.; Mahaux, M.; Penzenstadler, B.; Rodriguez-Navas, G.; Salinesi, C.; Seyff, N.; Venters, C. The karlskrona manifesto for sustainability design. arXiv, 2014; arXiv:1410.6968. [Google Scholar]
  5. Calero, C.; Piattini, M. Introduction to Green in Software Engineering. In Green in Software Engineering; Calero, C., Piattini, M., Eds.; Springer: Cham, Switzerland, 2015. [Google Scholar]
  6. Naumann, S.; Kern, E.; Dick, M.; Johann, T. Sustainable software engineering: Process and quality models, life cycle, and social aspects. In ICT Innovations for Sustainability; Springer: Cham, Switzerland, 2015; pp. 191–205. [Google Scholar]
  7. Penzenstadler, B. Infusing green: Requirements engineering for green in and through software systems. In Proceedings of the Third International Workshop RE4SuSy, Karlskrona, Sweden, 26 August 2014; pp. 44–53. [Google Scholar]
  8. Naumann, S.; Dick, M.; Kern, E.; Johann, T. The greensoft model: A reference model for green and sustainable software and its engineering. Sustain. Comput. Inform. Syst. 2011, 1, 294–304. [Google Scholar] [CrossRef]
  9. Moraga, M.Á.; García-Rodríguez de Guzmán, I.; Calero, C.; Johann, T.; Me, G.; Münzel, H.; Kindelsberger, J. GreCo: Green code of ethics. J. Softw. Evol. Process 2017, 29, e1850. [Google Scholar] [CrossRef]
  10. Carrega, A.; Repetto, M. Exploiting novel software development paradigms to increase the sustainability of data centers. In Proceedings of the 2016 IEEE/ACM 9th International Conference on Utility and Cloud Computing (UCC), Shanghai, China, 6–9 December 2016; pp. 310–315. [Google Scholar]
  11. Malmodin, J.; Lundén, D. The Energy and Carbon Footprint of the Global ICT and E&M Sectors 2010–2015. Sustainability 2018, 10, 3027. [Google Scholar]
  12. World Energy Outlook 2017 Report; International Energy Agency: Paris, France, 2017.
  13. Muñoz López, L.; Antón Martínez, P.; San Agustín García, A. Informe Anual del sector TIC y de los Contenidos en España. ONTSI, Ministry of Energy, Tourism and Digital Agenda, 2017. Available online: http://www.ontsi.red.es/ontsi/sites/ontsi/files/Informe%20anual%20del%20Sector%20TIC%20y%20de%20los%20Contenidos%202017.pdf (accessed on 15 November 2018).
  14. Manotas, I.; Bird, C.; Zhang, R.; Shepherd, D.; Jaspan, C.; Sadowski, C.; Pollock, L.; Clause, J. An empirical study of practitioners’ perspectives on green software engineering. In Proceedings of the 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE), Austin, TX, USA, 14–22 May 2016; pp. 237–248. [Google Scholar]
  15. Dick, M.; Drangmeister, J.; Kern, E.; Naumann, S. Green software engineering with agile methods. In Proceedings of the 2013 2nd International Workshop on Green and Sustainable Software (GREENS), San Francisco, CA, USA, 30 May 2013; pp. 78–85. [Google Scholar]
  16. Couto, M.; Borba, P.; Cunha, J.; Fernandes, J.P.; Pereira, R.; Saraiva, J. Products go Green: Worst-Case Energy Consumption in Software Product Lines. In Proceedings of the 21st International Systems and Software Product Line Conference-Volume A, Sevilla, Spain, 25–29 September 2017; pp. 84–93. [Google Scholar]
  17. Drangmeister, J.; Kern, E.; Dick, M.; Naumann, S.; Sparmann, G.; Guldner, A.; Birkenfeld, E.C. Greening Software with Continuous Energy Efficiency Measurement. In Proceedings of the 43rd GI-Jahrestagung 2013, Koblenz, Germany, 16–20 September 2013; pp. 940–951. [Google Scholar]
  18. Mohankumar, M.; Kumar, D.M.A. Empirical study on green and sustainable software engineering. In Proceedings of the 14th International Conference on Software Engineering, Parallel and Distributed Systems (SEPADS ’15), Dubai, UAE, 22–24 February 2015. [Google Scholar]
  19. Laskurain, I.; Ibarloza, A.; Larrea, A.; Allur, E. Contribution to Energy Management of the Main Standards for Environmental Management Systems: The Case of ISO 14001 and EMAS. Energies 2017, 10, 1758. [Google Scholar] [CrossRef]
  20. Garg, S.K.; Buyya, R. Green cloud computing and environmental sustainability. In Harnessing Green IT: Principles and Practices; John Wiley & Sons: Hoboken, NJ, USA, 2012; pp. 315–340. [Google Scholar]
  21. Ghamkhari, M.; Mohsenian-Rad, H. Energy and performance management of green data centers: A profit maximization approach. IEEE Trans. Smart Grid 2013, 4, 1017–1025. [Google Scholar] [CrossRef]
  22. Wu, J.; Guo, S.; Li, J.; Zeng, D. Big data meet green challenges: Big data toward green applications. IEEE Syst. J. 2016, 10, 888–900. [Google Scholar] [CrossRef]
  23. Zhu, C.; Leung, V.C.; Shu, L.; Ngai, E.C.-H. Green internet of things for smart world. IEEE Access 2015, 3, 2151–2162. [Google Scholar] [CrossRef]
  24. Santos, F.S.; Moura, H.P. What is wrong with the software development? research trends and a new software engineering paradigm. In Proceedings of the 24th ACM SIGPLAN Conference Companion on Object Oriented Programming Systems Languages and Applications, Orlando, FL, USA, 25–29 October 2009; pp. 895–900. [Google Scholar]
  25. Vegas, S.; Dieste, Ó.; Juristo, N. Difficulties in running experiments in the software industry: Experiences from the trenches. In Proceedings of the Third International Workshop on Conducting Empirical Studies in Industry, Florence, Italy, 16–24 May 2015; pp. 3–9. [Google Scholar]
  26. Escalona, M.; Lopez, G.; Vegas, S.; García-Borgoñon, L.; García-García, J.; Juristo, N. A Software Engineering Experiments to value MDE in testing. Learning Lessons. In Proceedings of the 2016 XXI Jornadas de Ingeniería del Software y Bases de Datos, Salamanca, Spain, 13–15 September 2016. [Google Scholar]
  27. Maimone, A.; Georgiou, A.; Kollin, J.S. Holographic near-eye displays for virtual and augmented reality. ACM Trans. Graph. (TOG) 2017, 36, 85. [Google Scholar] [CrossRef]
  28. Winters, T. Non-atomic refactoring and software sustainability. In Proceedings of the 2018 IEEE/ACM 2nd International Workshop on API Usage and Evolution (WAPI), Gothenburg, Sweden, 7 May–27 June 2018; pp. 2–5. [Google Scholar]
  29. Feranecová, A.; Ivaničková, M.; Jergová, N.; Sabolová, M. Benefits of EMAS easy implementation in SMEs. In Production Management and Engineering Sciences; ROUTLEDGE in Association with GSE Research, Taylor & Francis Group: London, UK, 2016; Volume 61, pp. 61–65. [Google Scholar]
  30. Women, UNIDO-UN. Sustainable Energy for All: The Gender Dimensions; UN WOMEN & United: New York, NY, USA, 2013. [Google Scholar]
  31. Prescriptions for Effective Implementation of the Sustainable Development Goals in Japan. Project on Sustainability Transformation beyond 2015. 2016. Available online: http://kanie.sfc.keio.ac.jp/post2015/wp-content/uploads/2016/02/SDGs-prescriptions_english.pdf (accessed on 15 October 2018).
Figure 1. Research methodology.
Figure 1. Research methodology.
Applsci 08 02312 g001
Figure 2. Comparison of results. Application of technical practices vs. the perceived helpfulness of technical practices in being more sustainable at work.
Figure 2. Comparison of results. Application of technical practices vs. the perceived helpfulness of technical practices in being more sustainable at work.
Applsci 08 02312 g002
Figure 3. Details on the sustainability of the software development process.
Figure 3. Details on the sustainability of the software development process.
Applsci 08 02312 g003
Figure 4. Form used to apply for funding from the INFO, in which ISO 14001 and EMAS are considered (excerpt).
Figure 4. Form used to apply for funding from the INFO, in which ISO 14001 and EMAS are considered (excerpt).
Applsci 08 02312 g004
Table 1. Technical practices analyzed in this study.
Table 1. Technical practices analyzed in this study.
#QPracticesSource
1Requirements reuseManotas et al. [14]
2Code reuse
3General sustainability policies (how the company operates)
4Specific sustainability requirements
5Principles or requirements for usability and user experience in the development of products
6Principles or requirements for accessibility in the development of products
7Principles or requirements of internationalization in the development of products
8Agile and lean methodologiesDick et al. [15]
9Software product linesCouto et al. [16]
10Testing and continuous integrationDrangmeister et al. [17]
11Standardization of practices, tools, technical solutions, etc.Mohankumar and Kumar [18]
12Certification in some sustainability standard (for example, ISO 14001 and/or EMAS)Laskurain et al. [19]
13Consideration of sustainability when software systems and services are purchased or outsourcedManotas et al. [14]
14Cloud computing and virtualization servicesGarg and Buyya [20]
15Use of “green” data center infrastructures (using renewable energies)Ghamkhari and Mohsenian-Rad [21]
16Big data and data miningWu et al. [22]
17Internet of thingsZhu et al. [23]
18Virtual reality and augmented realitySantos and Moura [24]
Table 2. Best practices for sustainability in the software development process (based on and adapted from Manotas et al. [14]).
Table 2. Best practices for sustainability in the software development process (based on and adapted from Manotas et al. [14]).
#QStatement
1The applications that I develop have energy use requirements
2I am willing to sacrifice performance, usability, etc., in exchange for having a lower energy use
3I take energy use issues into account in design
4When the use of energy is being evaluated, I take into account the different factors involved (use scenarios, environment, hardware, etc.)
5Energy use issues have an influence on the way I write code
6I know the use of energy of my code
7I could learn how to improve the use of energy
8The good use of energy should be the responsibility of the applications
9I learn about problems with the use of energy of my applications
10I want to learn about problems with the energy use of my applications
11Problems with the use of energy occur more frequently than other performance problems
12Problems with the use of energy are more difficult to discover, diagnose and/or fix than other performance problems
13When I modify the code, I make changes that I think will improve the use of energy
14I document the code with the changes that are made to improve the use of energy
15I find out how the changes I make have an impact on the use of energy
16During code reviews or other discussions, the use of energy is mentioned
Table 3. Demographic results.
Table 3. Demographic results.
Demographic Data Fields
GenderRole
Male81 (86%)Management17 (18%)
Female13 (14%)Administrative1 (1%)
AgeProject manager14 (15%)
<254 (4%)Systems manager2 (2%)
25–3430 (32%)Technician50 (53%)
35–4438 (41%)Other10 (11%)
45–5419 (20%)Seniority in the company
>553 (3%)0–1 years26 (28%)
Educational level2–5 years32 (34%)
Secondary/Vocational15 (16%)6–10 years24 (25%)
Graduate/Master69 (74%)>10 years12 (13%)
Doctorate5 (5%)Company size
Other5 (5%)Micro (<10 employees)17 (18%)
Small (10–49 employees)34 (36%)
Medium (50–249 employees)25 (27%)
Large (>250 employees)18 (19%)
Table 4. Summary of the results of the statistical tests.
Table 4. Summary of the results of the statistical tests.
Set#QProfileSize
Sig.OR, 95% CISig.OR, 95% CI
First10.0540.395, 0.151–1.0310.0180.328, 0.127–0.843
20.5691.567, 0.331–7.4200.8731.135, 0.240–5.374
30.1252.118, 0.803–5.5840.6420.801, 0.314–2.045
40.7950.860, 0.276–2.6800.0100.159, 0.033–0.754
50.0300.398, 0.171–0.9240.0760.474, 0.207–1.088
60.0090.324, 0.138–0.7630.1540.543, 0.234–1.261
70.0760.471, 0.204–1.0880.1540.543, 0.234–1.261
80.0170.357, 0.152–0.8390.5871.256, 0.551–2.862
90.7470.850, 0.316–2.2840.5610.743, 0.272–2.028
100.1230.500, 0.206–1.2130.0232.866, 1.142–7.193
110.3610.682, 0.300–1.5520.7531.141, 0.503–2.589
120.5840.704, 0.199–2.4880.9840.987, 0.279–3.489
130.8960.875, 0.118–6.4860.3950.381, 0.038–3.803
140.0440.419, 0.179–0.9840.0510.436, 0.188–1.012
150.2480.279, 0.028–2.7840.8611.195, 0.161–8.861
160.0000.089, 0.019–0.4210.8600.907, 0.307–2.681
170.7881.160, 0.393–3.4270.0170.219, 0.058–0.830
180.4840.429, 0.038–4.8950.6610.583, 0.051–6.663
Second10.2120.456, 0.130–1.5990.9750.981, 0.303–3.176
20.2600.449, 0.109–1.8570.3380.525, 0.138–1.997
30.5811.365, 0.451–4.1290.6261.321, 0.430–4.064
40.7631.171, 0.419–3.2750.5161.414, 0.495–4.038
50.4260.686, 0.271–1.7380.7920.884, 0.353–2.211
60.6170.797, 0.327–1.9420.9310.962, 0.396–2.334
70.9931.004, 0.427–2.3610.8740.933, 0.397–2.196
80.4540.671, 0.235–1.9150.3530.614, 0.218–1.727
90.6981.174, 0.521–2.6430.3170.660, 0.292–1.492
100.1871.910, 0.725–5.0370.6470.800, 0.307–2.082
110.7020.788, 0.231–2.6860.7510.822, 0.245–2.764
120.8111.109, 0.475–2.5910.9671.018, 0.435–2.383
130.3981.478, 0.596–3.6620.6090.789, 0.319–1.953
140.1430.499, 0.195–1.2750.3300.637, 0.257–1.581
150.2900.600, 0.232–1.5510.6421.249, 0.489–3.188
160.2821.565, 0.691–3.5450.1081.958, 0.859–4.461
170.6401.215, 0.538–2.7430.1270.527, 0.230–1.205
180.2810.611, 0.248–1.5010.1250.485, 0.191–1.233
Third10.9121.054, 0.416–2.6720.9921.005, 0.396–2.548
20.5261.366, 0.519–3.5950.3130.604, 0.226–1.616
30.7470.850, 0.316–2.2840.2770.568, 0.204–1.585
40.5340.753, 0.308–1.8430.0460.385, 0.148–1.000
50.7320.846, 0.325–2.2000.9750.985, 0.378–2.569
60.9100.947, 0.369–2.4290.4640.700, 0.268–1.824
70.7471.176, 0.438–3.1620.9401.039, 0.385–2.803
80.0930.495, 0.217–1.1280.2841.563, 0.689–3.541
90.8551.097, 0.407–2.9580.9400.963, 0.357–2.597
100.8840.931, 0.357–2.4280.9751.015, 0.389–2.649
110.3920.587, 0.172–2.0040.3491.789, 0.524–6.106
120.0300.398, 0.171–0.9240.7531.141, 0.503–2.589
130.5610.750, 0.284–1.9830.8451.102, 0.417–2.914
140.9901.007, 0.333–3.0440.9381.045, 0.346–3.161
150.8120.864, 0.257–2.9020.7610.827, 0.242–2.821
160.0810.338, 0.096–1.1890.2430.479, 0.136–1.680

Share and Cite

MDPI and ACS Style

García-Berna, J.A.; Carrillo de Gea, J.M.; Moros, B.; Fernández-Alemán, J.L.; Nicolás, J.; Toval, A. Surveying the Environmental and Technical Dimensions of Sustainability in Software Development Companies. Appl. Sci. 2018, 8, 2312. https://doi.org/10.3390/app8112312

AMA Style

García-Berna JA, Carrillo de Gea JM, Moros B, Fernández-Alemán JL, Nicolás J, Toval A. Surveying the Environmental and Technical Dimensions of Sustainability in Software Development Companies. Applied Sciences. 2018; 8(11):2312. https://doi.org/10.3390/app8112312

Chicago/Turabian Style

García-Berna, José Alberto, Juan Manuel Carrillo de Gea, Begoña Moros, José Luis Fernández-Alemán, Joaquín Nicolás, and Ambrosio Toval. 2018. "Surveying the Environmental and Technical Dimensions of Sustainability in Software Development Companies" Applied Sciences 8, no. 11: 2312. https://doi.org/10.3390/app8112312

APA Style

García-Berna, J. A., Carrillo de Gea, J. M., Moros, B., Fernández-Alemán, J. L., Nicolás, J., & Toval, A. (2018). Surveying the Environmental and Technical Dimensions of Sustainability in Software Development Companies. Applied Sciences, 8(11), 2312. https://doi.org/10.3390/app8112312

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop