PDF  PubReader

Woo* , Jeong** , and Huh***: Improvement of ITSM IT Service Efficiency in Military Electronic Service

Hanchul Woo* , Suk-Jae Jeong** and Jun-Ho Huh***

Improvement of ITSM IT Service Efficiency in Military Electronic Service

Abstract: IT Service Management (ITSM) achieves consolidated management for the IT services supporting the acquisition system, and no outside connections can be established with such ITSM. Issues pertaining to the D2B can be addressed to System Q&A or a Call Center for problem-solving. In other words, internal staff can take the necessary measures for problems by directly connecting with ITSM. Currently, diverse innovative technologies are being used in electronics and ubiquitous computing environments. This allows us to create a better world by providing the backbone for remarkable development in our human society in the fields of electronics, devices, computer science, and engineering. Following the expansion of IT services in the military acquisition sector such as Defense Electronic Procurement, military export/import support system, etc., customers’ dependence on IT for conducting business with the military or related companies is increasing, including the military’s dependence on the same technology for services to the public. Nonetheless, issues pertaining to the simplified/integrated management of complex IT service management systems, including slow system recovery, lack of integrated customer service window, and insufficient information sharing, have become the priority problems that IT managers are required to solve. Therefore, this study conducted research on the integrated management of IT services provided by Korea’s national defense acquisition system, which was developed based on the existing system IT Infrastructure Library (ITIL) v2, and investigated the level of satisfaction with services with focus on ensuring that it can be used for understanding the necessity of the system and its advancement in the future.

Keywords: Defense Acquisition Program , ITSM , Military Electronic Service , Satisfaction Level , Software Engineering

1. Introduction

The national defense electronic procurement system (D2B), national defense standard management system, and relevant homepages are used for military acquisition agencies/systems. Likewise, for the internal staff, portals, electronic documents, and office computer equipment are being used. Over the last few decades, the defense business has become more sophisticated, introducing innovative 21st century technologies, and defense contractors are consistently approaching companies with cutting-edge technologies regardless of their size. Digitalized and automated technologies have become integral sources of advanced modern weapon systems [1-3].

In most cases, however, the development and approval process for defense projects takes a long time, so these small/medium-sized weapon system development companies may not be able to keep their business until the actual implementation of their new systems.

It would be wise for innovative but small companies to partner with an established defense contractor to avoid any expected unnecessary and costly processes when dealing directly with the Ministry of Defense. This method allows them to be supported, guided, and funded as needed as long as the contract is valid while avoiding any unfamiliar bureaucracy.

For example, companies like techUK often assist small and medium-sized enterprises (SMEs) that are interested in participating in defense systems by offering some foundational support. According to techUK Chairman Tim Gibson, the strict regulations make it quite difficult for SMEs to participate in military supply chains especially when they wish to export their product or technology overseas, so it would be a good idea to establish a partnership with some multinational corporations that are familiar with the military markets in NATO, North America, Japan, and/or Australia [4-6].

Nowadays, the concept of ITSM is being discussed as a solution to issues pertaining to IT service management in businesses. The wider use of IT infrastructure has been found to make a company depend on the quality, volume, and availability provided by the ICT infrastructure [5,6]. In other words, the IT infrastructure is designed focusing on the technology rather than the rational, efficient customer-oriented process due to the biased view of “IT is the business” or “The business is IT”. Military agencies are following the same trend, paying increasing attention to ITSM, with the Defense Acquisition Program Administration (DAPA) of the Republic of Korea (ROK) proactively adopting the concept by systemizing it to apply to their operations.

The objective of the study is to establish transparency or fairness in the Korean armed services procurement systems and secure integrity, confidentiality, and availability through the construction of an appropriate ITSM to redefine, expand, and improve their services for the future. Moreover, by scientifically investigating the level of satisfaction before and after the construction of an experimental system model, the results will be positively reflected to the establishment of a future advanced procurement system.

2. Related Research

IT Service Management (ITSM) refers to an advanced IT infrastructure management system for managing all of the processes, resources, and technologies comprehensively to provide service quality at the agreed-upon level within the range of reasonable cost. Aidan Lawes, CEO of itSMF where global IT experts participate to discuss and share various IT practices to develop the current ITSM systems further, has defined ITSM as “one that includes all the activities associated with the entire life cycle related to developing and providing high-quality IT service”. Market research firm Gartner has also defined it as “an aggregate of processes, organizational capacities, and technologies necessary for providing a logical, predictable IT service” and suggested maximizing the business value of IT services, aiming to create the performance required at a given cost, as its objective. Through the application of a certain form of IT service, the user and the service will be able to achieve IT process standardization; by creating a clear communication channel based on common terminology and management indices, they are expected to be able to share the effects of cost reduction and service quality improvement [7,8].

Thus, ITSM is used not only as a tool for gathering the operating data of an IT operation room but also as a roadmap that consistently shows the future direction of IT services. At the itSMF, it was presented that ITSMs are being used in hospitals as a communication tool replacing the existing groupware, performing the same function, and they have earned raves from the participants. The question here is what the differences are between “communication” and “navigation”. The former is a major approach that can assist in business through the exchange of exact information, and many companies are developing and maintaining various types of processes or IT systems to achieve efficient communication. IT companies including Korea HP and IBM Korea are advancing their ITSM systems to gain an advantage over the others. For example, according to Yong-Seok Shin, who is working as a consultant for Korea HP, while their competitors are focusing on product sales and providing consulting services through channel partner firms, they are maintaining their own ITSM consultants at the level of over 200,000 at all times; this shows that they are capable of providing a comprehensive solution optimized for an individual customer’s demand. Similarly, IBM Korea has integrated their consulting team (10+ staff) and software team (5–6 staff), both of which were operated separately until last year, as a single virtual ITSM design team to unify their consulting, design, and operating capacities for their customers [9-11].

In addition, judging that new demand can be created by combining CMDB (Configuration Management Database) and existing ITSMs in terms of IT resource management, they are actively promoting the necessity of linking an ITSM system with CMDB and constructing an automated management system to maximize the effect of ITSM. For instance, when an ITSM is linked with a CMDM, the system failures reported by the customers can be processed rapidly and accurately. Korea CA and BMC Korea established a similar plan, i.e., to provide a solution optimized to individual companies by determining their respective target markets and improve IT efficiency through IT service automation. Based on these plans, Korea BMC Software approached the mainframe, Unix, and x86 markets in 2009 and attempted to achieve 115% sales growth by focusing on open system-based automation rather than mainframe automation for the banking industry. Meanwhile, Korea CA not only provided independent solutions but also concentrated on providing an integrated environment through the simplification of the IT management environment of each customer based on their global experience accumulated over the years under the Enterprise IT Management (EITM) strategy optimized for the business requirements of the customers [12-14].

The ITSM market, which had been growing by targeting major companies since 2005, expanded further after 2008 when it focused on small/medium-sized IT outsourcing firms including those operated by major companies themselves; thus accelerating the spread of ITSM systems in the SMB market. Companies that started to target the SMB market included LG CNS, Kolonbenit, and Daewoo Information System. At the same time, Dongbu CNI, Shinsegae IC, and Lotte IC attempted to enter the consulting market for the construction of IT service management systems [15-17].

ITSM is an IT service management framework which aims to prevent the deterioration of service quality due to diversified IT management environments, lack of standard management system, and unstable system operation, and provide the details of the standard IT service management process, organization, and technology based on ITIL (IT Infrastructure Library)—the best operation/management practice for securing higher-level service quality and competitiveness [18-21].

Fig. 1 shows that the most important elements such as manpower, process, and products (tools & technologies) should be used efficiently and effectively. As a universal definition of ITSM agreed upon by most of the people in the IT field, this is an approach for systematically managing the operation of information systems based on a management and enterprise-wide perspective rather than conventional technology-based management. That is, in a broad sense, ITSM is an approach for managing all the aspects (e.g., provision support) of IT service more systematically. Therefore, ITSM is similar to the concept of ‘IT Governance, Management of IT as a Business’.

As the best practice of ITSM, ITIL is a reference of the IT-related standard operation process for the companies. ITIL originates from the 45 books first published in 1986 by the British government agency, Central Computer and Telecommunication Agency (CCTA, currently Office of Government Commerce [OGC]) to provide guidance for systematically managing the information infrastructures of all the government agencies.

Fig. 1.

The Definition of ITSM.
1.png

Meanwhile, Fig. 2 shows the industries where the organizations have introduced ITSMs. The data shows that the companies involved in systems integration (SI) occupy the top of the chart, followed by financial and communications companies.

Fig. 2.

Introduction status by industry by ITSM year in the Republic of Korea.
2.png

By industry, companies operating in the IT, public, and financial sectors stay on top, specifically showing that over 50% of them belong to the IT or financial sector due to their heavy dependence on IT systems. Nonetheless, it can also be seen that ITSMs have currently spread to other areas including manufacturing or service industry. Due to the characteristics of the IT industry, most of the companies affiliated with a certain SI firm that has already introduced an ITSM system are classified as those that have done the same, so it is estimated that there are actually more of such companies; thus proving that the activation of ITSMs is being carried out gradually in our country.

Public institutions or major corporations are not the only ones introducing ITSM systems. The introduction of ITSM is no longer an option but a requirement for the military.

The strategic objectives of military organizations should be mutually connected with their IT service strategy, and they need to respond to changes in their organizational objectives effectively. They should also be able to prove the investment value of their expensive IT services to the department establishing the organizational objectives. Competitive IT services can lead not only corporate businesses but also military organizations and should appear consistently through such process. As the quality of the services ultimately determines the success or failure of businesses, it is now common sense that the simple operating angles of the past cannot guarantee the service quality required by modern businesses. The construction of an ITSM system allows the service providers to improve their immature level of IT operation/management in phases, ultimately strengthening their roles or functions to guarantee the service quality required by the organizational objectives.

Fig. 3 shows ITSM Necessity. It is essential that emphasis be placed on IT infrastructures as a conventional IT management concept, that users who invest or use IT as a customer be defined, that IT be considered a service that should be provided, and that the delivery process of such service as a repetitive and measurable process is now seen not as an option but a necessity for the military organizations. In other words, in terms of scientific and systematic organization management, IT management has emerged as an essential linchpin of IT services; thus further emphasizing the necessity of ISTM.

Fig. 3.

ITSM necessity.
3.png

3. Application of ITSM as an IT Service to the Military Acquisition Program

The objective of this kind of application is to propose a future research direction by analyzing the effects of IT service before and after the introduction of ITSM, which aims to provide various competent IT services to different customers involved in the military acquisition business.

3.1 Necessity of IT Service and Operation Management in the Military Acquisition Field

Currently, the IT services in the military acquisition field are divided into National Defense Procurement System (D2B), Defense for Business (D4B), Integrated Business Management System, Internet Homepage, and internal IT service. D2B is a military supplies acquisition service carried out through the electronic procurement system available to the public, wherein over 200,000 members are participating in all sorts of tenders for domestic/foreign procurements including facilities. Meanwhile, D4B provides an IT service for information related to the import/export of defense materials, whereas the Integrated Business Management System is being used for total (integrated) business management for weapon systems acquisition. In addition, the homepage of DAPA is implementing IT services that provide information related to defense-related businesses to both the military and the public. Its internal process includes requests for PC maintenance, customer support, IT-related maintenance, consumables, or administrative support, in addition to providing information associated with all kinds of security incidents.

3.2 Analysis of Application of ITSM as a Military Acquisition IT Service

Although the IT services for the military acquisition sector started at the same time as the opening of DAPA in January 2006, their early services were either insufficient or not adequately updated, providing the services independently instead of an integrated service. Moreover, since they also handled customers’ varying requests individually through their own systems and managed the data separately, it was impossible to check their operating statuses comprehensively. As a result, the efficient management of operating personnel and utilization of valuable data were not achieved as they had initially intended. The IT services prior to the introduction of ITSM are represented as “the dispersed points for complex and diversified systems, operating organizations and supporting environments, and customer service requests.”

Many changes have been made since the introduction of ITSM in 2009, the biggest of which is “Integrated IT service management”, which allowed the flexible use of manpower following the improved operational efficiency and reduced processing time based on a single point of contact (i.e., unifying all the systems into an IT service desk) for all the requests and information of accumulated data.

Fig. 4 presents the ITSM system process procedure (fault processing) as a feature of the system service after the introduction of ITSM as “information of unified systems/data”. The analysis revealed that the greatest effect of ITSM was the “completion of an integrated support infrastructure” based on increased management efficiency, improved service reliability, and expeditious business support.

Meanwhile, the IT service desk described in Fig. 5 allows quickly responding to IT-related problems through a user interface that shows the overall status of requests for troubleshooting.

Fig. 6 presents an example of requests received through the IT service desk of DAPA in each specified period and their follow-ups. These can be used as a data point for future response to IT-related problems. Also, Table 1 shows the performance after the introduction of ITSM.

Fig. 4.

ITSM system process procedure (fault processing).
4.png

Fig. 5.

IT service desk (ITSM-supported) of DAPA.
5.png

Fig. 6.

Received requests through the DAPA IT service desk.
6.png

Table 1.

Performance after the introduction of ITSM
Category Performance index Before the introduction (year 2009) After the introduction (year 2012) Present
Improved customer satisfaction Impatience prob. (%) 30 4 1
Satisfaction level (max: 5) 3 4.5 4.7
Improved internal operating efficiency Processing time:[TeX:] $$1^{\mathrm{st}}$$call (min) 50 5 3
Primary processing rate (%) 60 95 98
Monthly processing cases 1,000 7,000 5,000

The data prior to introduction is an average estimation.

Moreover, in terms of the efficient operation of IT service, the quantification of IT service was not easy before the introduction, and it was difficult to check customers’ attitude toward the call center and impatience probability (i.e., giving up consulting while waiting for their turn). In other words, sharing information about IT service requests did not perform well. The data shows that the impatience probability has decreased by 1% from 29% during the pre-introduction period, whereas the customer satisfaction level has increased from 60% to 95%. In addition, the average primary processing time was less than 3 minutes compared to 50 minutes before the introduction. In general, the primary request processing performance increased from 60% to 95% even though the number of processing request calls increased.

According to the analysis, after the introduction of ITSM, the reliability of IT service for customers has increased thanks to the swift handling of problems and sharing of information, which in turn has led to self-care measures without making unnecessary calls, increased level of focus on individual tasks, cost reduction, and decreased repetitive work on the same task. These have eventually resulted in increased efficiency in the IT service.

3.3 Data Processing by ITSM User Interface in UML

Fig. 7 shows the data processing presented in unified modeling language (UML) of user interface for access to manage the ITSM proposed in this paper systematically. The process can be divided into data allocation, data rearrangement, data connection, data loading, and data analysis.

As mentioned above, the key elements of ITSM are the IT resources and organization for the IT service management and part of ITIL, which is a set of best practices standardizing the work process. The service management presented by the ITIL framework can be divided into two elements: service delivery and service support. Service delivery is the process of setting and managing the service level to provide the IT service to help the enterprise achieve the business goal, and service support is the process of supporting service delivery. Fig. 8(a) shows the main processes in UML. In Fig. 8(b), API operates in five sequences: change, incident, release, configuration information, and problem.

Fig. 7.

Data processing by ITSM user interface.
7.png

Fig. 8.

Contents of the main process of ITSM proposed. (a) The ITSM process (release, configuration information, problem) and (b) the ITSM Process (change, incident).
8.png

Fig. 9 shows the UML presentation of the hierarchical mechanism in the ITSM user area. The purpose of ITSM is to establish the IT service management system and to save costs, increase customer satisfaction, assure the service level, and improve services continually. Layer 2 has its own API to interface with the system, and layer 3 interchanges resources for it. The data processing mechanism of the proposed ITSM system can be divided into four types.

Fig. 9.

Hierarchical mechanism in the ITSM user area.
9.png

Fig. 10 shows the service desk UML of the ITSM system proposed in this paper. It is presented in four APIs with an input button.

For the future development direction of ITSM, the underlying infrastructure and document information system should be integrated for management while improving the operating/management condition of IT assets to achieve high-quality information management. Likewise, for the quantification of IT service, all the elements in the service should be quantified for use to improve/allow efficient distribution of IT operating resources and increase the customer satisfaction level by tracking customers’ IT service requests on a real-time basis. Achieving such goals will fulfill the primary objective of DAPA (i.e., securing enhanced reliability and transparency) for both the military and the public, but it is also necessary for continuous investment to be made for the improvement. In the study “Routinization of IT Service Management: A Descriptive Case Study” [20], the author explained that merely adopting an ITSM system is not enough to achieve an efficient ITSM, but that the synergy effect of ITSM can only be achieved through continued support from the CEO and improved/systematic management [1-4]. In other words, the adoption of ITSM does not implicitly guarantee successful operational management; instead, the CEO’s will and reflection of performance outcomes to the management process are the main factors in establishing an effective, efficient ITSM system.

Fig. 10.

Data processing mechanism of the proposed ITSM system: (a) incident process, (b) problem management, (c) change management, and (d) release management.
10.png

4. Conclusion and Future work

In this study, the effects of the introduction and application of ITSM to increase efficiency in IT service for the military acquisition sector have been analyzed. Along with military supplies and weapon systems procurement/acquisition, this sector is a vital national service with a variety of special IT services supporting business. After the introduction of ITSM, the IT services dispersed throughout each acquisition sector have been integrated into a single IT service to solve problems pertaining to the previous services offered individually. As the biggest achievement, however, a framework that allows effective IT service support has been created. Nonetheless, it is still necessary to place ITSM at the center of the IT service in the military acquisition sector by developing the ITIL, which is insufficient in the existing ITSM system, and to utilize it actively for the achievement of higher level of customer satisfaction and organizational ROI (return on investment). First, for the relationship between service quality components and service value, human quality and resulting quality were found to have significant influence on service value, whereas system quality had none. This would mean that service value is not associated with the information management, color, or design of the system; upon actually examining the items included in measuring the system quality, they consist of “reliability in security”, “convenience in service use”, etc. From this perspective, only the failovers are worthy of time and effort in terms of continuity/availability/efficiency of the task (business), so service value cannot be found in the information management, colors, fonts, and designs of the site being analyzed. Second, for the relationship between service quality components and service satisfaction, the result showed that human quality and system quality included as a process quality had no significant positive influence on service quality. Such result, which varies from the existing one, was deemed to have stemmed from the perspective that the service request for solving a task-related problem is enough, so any consideration for service satisfaction is unnecessary. Still, service quality was also found to have an indirect influence on service satisfaction through an intermediate variable. Third, for the relationships between service value, service satisfaction, and service reliability, service value affected service satisfaction directly as expected but had no direct effect on service reliability. Meanwhile, service satisfaction did have a direct influence on the service itself, and such result shows that a Defense Acquisition Program Administration employee who values the ITSM service is usually satisfied with the service and trusts it very much. In conclusion, the key factor in service quality that actually gains trust is service value, on which both human quality and resulting quality have a direct influence, whereas the decisive parameter for enhancing service reliability is service satisfaction. These results can be useful when attempting to improve service quality through supplementary measures or improvements.

This study targeted the employees of the Defense Acquisition Program Administration; under the unique environment that placed heavy emphasis on security, it was quite difficult to extend the range of the research. Our future task is to conduct the same research on the ITSM service satisfaction for the other public institutions and compare the results with existing research works. We expect such work to lead us to a more reliable, valid conclusion that can be useful in providing a customized service depending on the characteristics of individual public institutions.

The future work task involves identifying the service quality factors in accordance with the characteristics of individual public institutions and applying them to the system design to construct a system that is able to provide a high level of satisfaction or reliability. Such work can be useful as basic information when modifying/enhancing/sophisticating the system to achieve more efficient business operation. Further study is required to identify the service quality factors in various types of public institutions and evaluate the level of satisfaction with their respective service systems based on the service value and the impact of satisfaction.

Acknowledgement

This research was supported by Energy Cloud RD Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science, ICT (No. NRF-2019M3F2A1073385). Also, this work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2017R1C1B5077157).

Biography

Hanchul Woo
https://orcid.org/0000-0003-1585-5792

He received B.S. Korea Army Academy at Yeong-Cheon (KAAY), Republic of Korea in Feb. 1998. He received Master of Business Administration (Military MIS), Dongguk University. 2005. Also, He received Ph. D. at Dept., of Defense Acquisition Program, Kwangwoon University, Seoul, Republic of Korea in Feb. 2020. Defense Acquisition Program Administration Service at ROK (2006-2019). Since 2019, he has been an Instructor/Assistant Professor of Defense Information Officer of Defense Security Support School (DSSS), Defense Security Support, Command, Gwacheon City, Republic of Korea.

Biography

Suk-Jae Jeong
https://orcid.org/0000-0001-7081-7567

He received the Ph.D. degree in industrial engineering from Yonsei University. He is an Associate Professor of operation management and supply chain with School of Business, Kwangwoon University. He has published numerous articles in research publications, including the International Journal of Advanced Manufacturing Tech- nology, Expert Systems With Applications, the International Journal of Production Economics, Energy, and Energy Policy.

Biography

Jun-Ho Huh
https://orcid.org/0000-0001-6735-6456

He was born in Kyoto, Japan. He was finished the Cooperative Marine Science and Engineering Program, Texas A&M University at Galveston, United States of America in Aug. 2006. Received B.S. in Science Degree from Department of Major of Applied Marine Sciences (Currently Faculty of Marine Biomedical Sciences). And, B.S. in Engineering degree (Double Major) from Department of Major of Computer Engineering from Jeju National University at Ara, Jeju, Republic of Korea in Aug. 2007. He received M.A. in Education Degree from Department of Major of Computer Science Education, Graduate School of Education, Pukyoug National University at Daeyeon, Busan, Republic of Korea in Aug. 2012. Received the Ph.D. in Engineering Degree from Department of Major of Computer Engineering, Graduate School, Pukyoug National University at Daeyeon, Busan, Republic of Korea in Feb. 2016. Since September 2019, he has been an Assistant Professor (Tenure Track) with the Department of Data Informatics, (National) Korea Maritime and Ocean University, Republic of Korea. His research interests include green IT, smart grid, edge computing, network security, the IoT, AI, big data, and system architecture.

References

  • 1 J. E. Stanley, R. F. Mills, R. A. Raines, R. O. Baldwin, "Correlating network services with operational mission impact," in Proceedings of 2005 IEEE Military Communications Conference, Atlantic City, NJ, 2005;pp. 162-168. custom:[[[-]]]
  • 2 I. Dimou, H. Wietgrefe, M. van Selm, L. Janovszki, "Demonstration of a cross security domain service management capability for federated missions," in Proceedings of 2014 IEEE Military Communications Conference, Baltimore, MD, 2014;pp. 184-191. custom:[[[-]]]
  • 3 U. Pathmanand, "Globalization and democratic development in Thailand: the new path of the military, private sector, and civil society," Contemporary Southeast Asia, vol. 23, no. 1, pp. 24-42, 2001.custom:[[[-]]]
  • 4 M. M. K. Man, "The relationship between distinctive capabilities, innovativeness, strategy types and the performance of small and medium-size enterprises (SMEs) of Malaysian manufacturing sector," Inter-national Business & Economics Research Journal (IBER), vol. 8, no. 11, pp. 21-34, 2009.custom:[[[-]]]
  • 5 D. J. Shyy, "Military usage scenario and IEEE 802.11s mesh networking standard," in Proceedings of 2006 IEEE Military Communications Conference, Washington, DC, 2006;pp. 1-7. custom:[[[-]]]
  • 6 O. Balci, W. F. Ormsby, "Network-centric military system architecture assessment methodology," International Journal of System of Systems Engineering, vol. 1, no. 1-2, pp. 271-292, 2008.custom:[[[-]]]
  • 7 V. B. Grasso, 2006;, http://www.nfaoa.org/documents/RL31686.pdf
  • 8 T. C. Wen, Y. C. Chang, K. H. Chang, "Cost-benefit analysis of RFID application in apparel retailing for SME: a case from Taiwan," Transportation Journal, vol. 49, no. 3, pp. 57-66, 2010.custom:[[[-]]]
  • 9 F. V. Mjelde, K. Smith, "Performance assessment of military team-training for resilience in complex maritime environments," in Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 2013;vol. 57, no. 1, pp. 2116-2120. custom:[[[-]]]
  • 10 C. P. Simion-Melinte, "Reconversion of military sites into business incubators and business support centers-the European experience," Business Excellence and Management, vol. 2, no. 2, pp. 79-86, 2002.custom:[[[-]]]
  • 11 Z. J. Acs, R. Morck, J. M. Shaver, B. Yeung, "The internationalization of small and medium-sized enterprises: a policy perspective," Small Business Economics, vol. 9, no. 1, pp. 7-20, 1997.custom:[[[-]]]
  • 12 W. A. Demmer, S. K. Vickery, R. Calantone, "Engendering resilience in small-and medium-sized enterprises (SMEs): a case study of Demmer Corporation," International Journal of Production Research, vol. 49, no. 18, pp. 5395-5413, 2011.custom:[[[-]]]
  • 13 C. M. Olszak, E. Ziemba, "Critical success factors for implementing business intelligence systems in small and medium enterprises on the example of upper Silesia, Poland," Interdisciplinary Journal of InformationKnowledge, and Management, vol. 7, no. 2, pp. 129-150, 2012.custom:[[[-]]]
  • 14 T. Gulledge, "B2B eMarketplaces and small-and medium-sized enterprises," Computers in Industry, vol. 49, no. 1, pp. 47-58, 2002.custom:[[[-]]]
  • 15 T. Hashino, "The rise and growth of small and medium-sized enterprises in industrial districtsi," Japanese Research in Business History, vol. 24, pp. 53-75, 2007.custom:[[[-]]]
  • 16 P. Devi, "E-governance for small and medium enterprises in a developing country like Fiji: potentials and problems," in Critical Thinking in E-Governance. New Delhi: Gift Publishing, pp. 235-241, 2007.custom:[[[-]]]
  • 17 L. X. Cunningham, C. Rowley, "The changing face of small and medium-sized enterprise management in China," in The Changing Face of Management in China. New YorkNY: Routledge, pp. 125-148, 2010.custom:[[[-]]]
  • 18 S. Park, J. H. Huh, "Effect of cooperation on manufacturing it project development and test bed for successful industry 4.0 project: safety management for security," Processes, vol. 6, no. 7, 2018.doi:[[[10.3390/pr6070088]]]
  • 19 H. Woo, S. J. Kang, B. K. Lee, "An analysis on ITSM application to improve the effectiveness of IT service in defense acquisition sectors analysis of factors for strengthening submarine industry basis," in Proceedings of the Institute of Control, Robotics and Systems Conference, 2012;pp. 249-258. custom:[[[-]]]
  • 20 H. Woo., "A study on the influence of quality factors of ITSM service on service trust in military acquisition institutions," Ph.D. dissertationKwangwoon University, Seoul, Korea, 2020.custom:[[[-]]]
  • 21 H. Woo, S. Lee, J. H. Huh, S. Jeong, "Impact of ITSM military service quality and value on service trust," Journal of Multimedia Information System, vol. 7, no. 1, pp. 55-72, 2020.custom:[[[-]]]