ARTÍCULOS ORIGINALES
DOI: 10.17230/ad-minister.29.7
Project maturity evaluation model for SMEs from the software development sub-sector
Modelo de evaluación de madurez en gestión de proyectos para pymes del subsector desarrollo de software
ÁLVARO JULIO CUADROS LÓPEZ1, CAROLINA GALINDRES, PAOLA RUIZ2
1 Universidad del Valle, Cali, Colombia. Correo electrónico: alvaro.cuadros@correounivalle.edu.co
ORCID: http://orcid.org/0000-0001-9248-3721
2 Universidad del Valle, Cali, Colombia. Correo electrónico: carolina.gbernal@gmail.com
ORCID: http://orcid.org/0000-0002-0339-0871
3 Universidad del Valle, Cali, Colombia. Correo electrónico: paola.ruiz@correounivalle.edu.co
ORCID: http://orcid.org/0000-0001-7086-3600
JEL: M1, M15
Recibido: 08/02/2016 Modificado: 14/03/2016 Aceptado: 02/05/2016
RESUMEN
El propósito de este artículo es presentar una propuesta de diagnóstico de la gestión de proyectos para pymes dedicadas al desarrollo de software. El modelo se fundamenta en el modelo de madurez de capacidades (CMMI-DEV) y el método de evaluación SCAMPI (por sus siglas en inglés). La propuesta considera una escala cuantitativa de satisfacción, valoración redundante de evidencias y criterios múltiples de selección de expertos. La propuesta fue probada con un caso de estudio en el que se evaluó la madurez en gestión de proyectos en una mediana empresa del sector de tecnologías de información y comunicaciones. El modelo concluyó que la compañía no alcanza el nivel 2 de madurez, pero se logró identificar que la organización ya tiene implementados el 92% de los procesos de ese nivel y el 77% del total de los procesos, lo que permite que la organización oriente sus esfuerzos de mejoramiento.
PALABRAS CLAVE CMMI; desarrollo de software; gestión de proyectos; modelos de madurez; SCAMPI.
ABSTRACT
The purpose of the paper is to present a project management maturity model for SMEs oriented to software development. The proposal is based on CMMI capability maturity model, and the SCAMPI evaluation method. The proposal includes a quantitative satisfaction scale, redundant evidence assessment, and multiple criteria for selecting experts. The proposal was validated with a case study carried out in a medium-sized company from the Information and Communications Technology sector. The model concluded that the company did not reach maturity level 2; however it showed that 92% of the processes from maturity level 2 and 77% of the total process had already been implemented, which allows the company to adopt a specific orientation for its improvement efforts.
KEYWORDS CMMI; software development; project management; maturity models; SCAMPI.
INTRODUCTION
The phenomenon of software development globalization has witnessed remarkable growth, with countries seeking to increase their export earnings via their software development industries (Iyidogan, 2014). The growth of national economies is based on software industries made up mainly of small and medium software companies.
Developing software capabilities in latecomers is not an easy task and one important concern of software industry companies has been the management of technology and innovation, and the development of software products with optimum use of resources, costs and time; in other words, the quest is to be efficient (Garzas, et al., 2013; Iyidogan, 2014)
Being efficient is not easy. The CHAOS report which is presented by the Standish Group every two years is one of the best known reports on success and failure statistics of IT projects. According to the 2013 report, the number of successful projects has increased but only from 37% to 39%, from 2010 to 2012 (The Standish Group International, 2013). For the purpose of this report, successful projects are those that were delivered on time, within budget and accomplished the required capabilities. The same report shows a high percentage of challenged projects, 43%, increased from 42% since 2010. Challenged projects are those that were completed late, over budget, and/or with less than the required features and functions.
In addition, failed projects have decreased but only from 21% to 18% over the same period of time. Failed projects are those which were cancelled prior to completion or delivered and never used. If we consider that 61% of projects are not fully successful, it can be understood that project management discipline has a challenge to provide efficiency. According to the study, the increase in success rates is due to a variety of best practices for project management and the use of better development tools and methods.
The scope of the research is important. About 60% of the projects in the study were from the USA, 25% were Europeans and the remaining 15% from the rest of the world (The Standish Group International, 2013).
Software industry in Colombia follows the pattern described below. Many of the companies are small and medium-sized, therefore some are unaware of management systems, which leads to a lack of specialism and maturity with regard to project management. Among their weaknesses, they have high project costs, long delivery times, products of insufficient quality and limited organization management if compared with other countries (Valencia, Villa, & Ocampo, 2009).
Similarly, the Colombian TICs Ministry has reported that out of an estimated 2,000 companies, 55% are micro, 34% are small and 9% are medium size organizations with low technological specialization (Ministerio de Tecnologías de la Información y las Comunicaciones, 2013b).
In Colombia, the information technology industry is considered strategic because it can continuously transform the economic and social environment (Valencia, Villa & Ocampo, 2009; Ministerio de Comercio Industria y Turismo, 2013). It is estimated that by 2032 the industry would represent 1.6% of Gross domestic product (GDP ), similar to what crude oil represents in today´s economy (Benavides et al. 2011). This is why the Colombian government is trying to strengthen competitiveness and productivity conditions. In order to create value in the industry and to improve its visibility as an attractive destination to business software, governments have been promoting the adoption of international standards, certifications or best practices models (Valencia, Villa & Ocampo, 2009; Ministerio de Comercio Industria y Turismo, 2013).
In 2007, the Colombian government launched the program ''Support to the strengthening of the national capacity of Software'' which carried out consultancy and accompaniment in the implementation and institutionalization of the CMMIDEV model in 55 SMEs in the software sector in different areas of Colombia (Villalba & Estrada, 2011). This began with the selection of companies, establishing and evaluating of legal, administrative, technical and operational criteria, which failed to reach at least 50% of the score. This showed the shortcomings in the organizational strategy and operations through engineering practices.
It was noted that, at the national level, an average score superior to one and a half point cannot be achieved for each revised practice group, with three being the maximum score for each group (Villalba & Estrada, 2011). This validated the need for software sector to improve its processes, with an improvement plan that included training in the CMMI-DEV model, use of technological tools, work equipment, and training on tools and methods of quality assurance. The final evaluation of the program, looking for the degree of institutionalization and adherence of the model showed that all areas were over 75%, which represents a good indicator.
Those efforts, however, are not sufficient. A field research performed in 2012 by the Colombian Government, found that about 50% of IT companies are not using any certification. And only 23% of them have ISO 9001 certification and 12% have CMMI. The study identified a high correlation between certification and company size, the larger the company the greater the chance to be certified. As an explanation, the study reported that obtaining certifications is expensive and can only be done by the biggest companies (Ministerio de Tecnologías de la Información y las Comunicaciones, 2012).
This pattern has been found in other industrial sectors in Colombia. Research about project management maturity in other industries has also been developed in Colombia showing low maturity levels. In addition, they show that company size is related to that maturity level. A research, carried out merely as a diagnostic study, was developed in 2008, with a group of medium-sized businesses (Aguirre & Córdoba, 2008). A total of 61 medium sized companies from biggest cities of the country were studied. The survey was answered by general managers and showed a low process maturity level. Missing processes of standardization and business process improvement tools were discovered. Later, in 2010, a similar research was developed with 5,187 companies from different industries in Bogotá (Arce & López, 2010). Although the maturity level in those companies was low, it increases with the size of the company.
Those efforts are seen not only in Colombia. One important concern of software industry companies has been the development of software products with an optimum use of resources, time and costs (Garzás et al., 2013); in other words, the quest is to be efficient. In this respect, small companies need efficient software engineering practices that are suitable for their particular characteristics to be more competitive (Garzás et al., 2013). Due to the impact over the efficiency of the product development process and therefore over the competitiveness, in recent years, a key research area in software engineering has been the evaluation of the maturity of software processes (Ahmed & Capretz, 2011).
For instance, (Chevers, 2014) evaluated project maturity in software companies in four Caribbean countries, Barbados, Guyana, Jamaica and Trinidad in order to find the factors which could produce higher quality software products.
Higher maturity enables a global understanding and a better vision of the workload and, as a result, a more consistent and repeatable work. Moreover, technical skills of the staff are improved and the use of technology may be maximized. When the productivity and efficiency of development activities are improved, the organization can develop, maintain and deliver high quality products, meeting business objectives and obtaining a higher customer satisfaction (Mesquida, Mas, Amengual, & Calvo-Manzano, 2012).
In order to increase maturity, companies have to engage in Software Process Improvement (SPI) programs which aim to understand the software process as it is used within an organization and thus drive the implementation of changes to that process to achieve specific goals such as achieving higher product quality or reducing costs. SPI models can assist companies in this regard and purport to represent beacons of ‘best practice' (Coleman & O'Connor, 2008).
Organizations adopt Software Process Improvement (SPI) frameworks to achieve more effectiveness in their development process, a higher quality of the final software product, and a reduced cycle time of product development (Shih & Huang, 2010; Allue, Dominguez, Lopez, & Zapata, 2013)
SPI models have been highly publicized and marketed, however they are not being widely adopted and their influence in the software industry therefore remains at a theoretical level rather than practical (Coleman & O'Connor, 2008).
In 2004, Jiang et al. (2004) made a research on relationship between software development processes maturity and project performance, by applying a survey to 160 software managers, project leaders and software professionals. Results showed that project performance can be improved through implementation of best practices in some specific areas. They also concluded that companies which adopt models such as the capability maturity model (CMM) tend to have substantially higher quality software, in a faster time and with higher productivity in its development.
Since then, different studies have focused on this maturity model. In 2010, (Von Wangenheim et al., 2010) conducted a study on the relationship between best practices promoted by two models leaders, the Project Management Base of Knowledge (PMBOK) and the Capability Maturity Model Integration (CMMI). They analyzed similarities and differences in management practices of each of them, to find most representatives. They concluded that it is possible the integration and harmonization of these models could help evaluate and put in ractice project management processes in an effective and efficient way (Von Wangenheim et al., 2010).
In addition, (Jamaluddin, Chin, & Lee, 2010)the greater it enhances project performance. However it is vital to first understand the requirements of such a maturity model in the context of the target industry or sector. In this study, we aim to assess the current status of project management maturity model (PMMM studied the adoption of project management maturity models in ICT companies and how this adoption helps them for better performance in their projects. They conducted surveys in different industries in Malaysia, finding that 42% of the participants implemented the PMMM model, 24% the CMMI, 6% the OPM3, 6% P3M3 and other industries do not use a specific model.
The primary reason respondents mentioned the adoption of PMMM model was to improve project management processes, since they considered that they allowed them to obtain a better outcome in their projects and organizations. However the study also shows that the adoption of PMMM in small enterprises is lower in comparison with large firms since its implementation is more expensive.
Following the same idea, Ehsan et al. (2010) conducted an investigation on the CMMI and SPICE models, in order to determine which of them is most often used by industries. The study found that the majority of software companies employ process improvement models, such as the CMMI model, while only a few use the SPICE model. Among the reasons mentioned were the fact that the CMMI model is open access, while the SPICE model is not, in addition to a greater availability of resources and professionals trained in CMMI.
The same study showed that the implementation of CMMI model is relatively more feasible in comparison with the SPICE model regarding their application. There have been better results in software companies that have implemented the CMMI model to manage their projects. In conclusion, the project managers are more satisfied with the results provided by the CMMI, without mentioning that customers feel greater confidence towards a company that implements the CMMI model.
Finally, a study, performed by (Selleri et al., 2015) synthesize, and present results on the use of the Capability Maturity Model Integration (CMMI researched the use of the CMMI model in combination with agile software development as increasing research topics in software literature. According to the authors CMMI is a maturity model used by many organizations all over the world, and its use has been one of the factors related to agile development. They also concluded that agile methodologies alone were not sufficient to obtain the maturity level desired, as resort to additional practices proves necessary.
In addition (Garzás et al., 2013), CMMI is the most popular and most widely used model in the world and for some time now it has been the only option for a certification of this type. However it has some limitations and many firms do not adopt CMMI because of its high costs.
The fact that it is inappropriate for small-scale organizations (which make up the greater part of the software industry) is another disadvantage. Moreover, this model takes a long time to implement (Garzas et al., 2013; Allue et al., 2013). In the following section, the base of CMMI and SCAMPI is presented as well as additional data proposed by authors.
METHOD
The study required to take some vital decisions prior to field research. Selecting the maturity model, designing the evaluation structure, and deciding on criteria for the selection of experts were important issues in the project.
First of all, it is important to decide which project maturity model to use. It is considered very important for a small company, not to start an assessment project until different models or proposals have been analyzed and understood according to the specific nature of the company.
Secondly, it is necessary to design the assessment structure which includes evaluation criteria, evaluation scale and the research instrument. Literature review allows us to identify project maturity models that are well known and receive a lot of publicity, however their structure is not completely published. Every research reviewed had its own structure and for the purpose of our study every item had to be decided depending on the kind of company as it is the goal of the proposal. Finally, the instrument, in an interview way, has to be applied and the selection of experts for that is very important. In this case, the proposal includes some criteria in order to be sure reliable information is used.
Selecting a maturity model
For the research, the Capability Maturity Model Integration proposal was selected following previous studies reported it as the most used in software industry and also its advantages over other proposals. This model was developed by Software Engineering Institute (SEI) with other organizations to be a best practice guide and facilitate maturity and capacity process evaluation (CMMI Institute, 2011) including both internal process improvement and external capability determinations. SCAMPI satisfies all of the Appraisal Requirements for CMMI (ARC).
The CMMI v1.3 works on improving three dimensions: product development processes (CMMI-DEV), product acquisition processes (CMMI-ACQ), and services processes (CMMI-SVC). Considering that the scope of the research is software development projects, it was decided to use the CMMI-DEV because it is the dimension that guides measurement, monitoring and management product development. Four categories belong to that dimension: Support, Project Management, Engineering and Process Management. As the scope of the research is Project Management, this category was used to assess the maturity. Seven processes areas are associated to this category and are grouped by maturity level, from level 2 to level 4 (CMMI Institute, 2013) as shown in Table 1:
Processes areas are good practices that companies should implement for every dimension. Every area is structured over goals, practices and sub-practices in a hierarchical structure. For the project, 261 sub-practices were valuated to assess the implementation level of 59 practices, 15 goals and 7 process areas.
Additionally, to consider an area as satisfied, all goals, practices and sub-practices have to be accomplished. Therefore, reaching the maturity level 4 means that all process areas for levels 2, 3 and 4 have been implemented. Part of the hierarchical structure of process areas, goals and practices can be seen in Table 2.
Designing the evaluation method
For the Project, it was decided to use Standard CMMI Appraisal Method for Process Improvement (SCAMPI), an evaluation method developed by SEI to complement to CMMI, which is very accepted and used in CMMI evaluations (CMMI Institute, 2013).
There are three types of SCAMPI which can be used for the evaluation, A, B and C class. For the project SCAMPI B-class was selected. In this class evaluation, the team has to be made up of two persons minimum and it is not necessary to have a certified team leader. In addition, evaluation has to be performed by two types of evidence, direct and affirmations. Direct evidence is tangible outputs results of performing a practice such as documents, derivables and products. Affirmations are oral or written confirmation of performing a practice such as interviews and conferences.
The maturity model provides evaluation categories however it does not provide the scale. The categories are Fully implemented (FI), Largely Implemented (LI), Partially Implemented (PI) and Not Implemented (NI). For the project, as a complement to the maturity model, authors designed a proposal of quantitative scoring rule in order to facilitate data analysis and a color code to make the visualization of the results easier. First of all, sub-practices can receive a score from 0 to 2 according to its level of implementation showed by evidence. As the model is a hierarchical structure, score of every component results from the average score of all elements inside it. It has to be clear that the scoring rule and the color code were defined by the authors with no preference. Color code and scoring rule for all evaluation categories are shown in Table 3.
Identifying experts
As the evaluation is based on the perception of participant, the selection of those people is important to have reliable information. However, the literature review did not provide details. Experts have been identified as people who know the projects, the content of the products being provided, or the assessment model (CMMI Institute, 2011)including both internal process improvement and external capability determinations. SCAMPI satisfies all of the Appraisal Requirements for CMMI (Von Wangenheim et al., 2010; Ensslin, Scheid, Ensslin, & Lacerda, 2012; Perez Mergarejo, Perez-Vergara, & Rodriguez-Ruiz, 2014).. For the project, some criteria were proposed to select the people for taking part in the research as an expert. Three experts were selected following the four criteria shown in Table 4.
Collecting information
In order to maintain the hierarchical structure of the model and clarify the process for the expert, field research was performed in three stages and every one required some meetings. In the first phase, objectives of the project were communicated to the general manager of the company and his group of collaborators. In addition, it was necessary to know the characteristics of the company, its projects and its processes, and also to identify the experts for the research.
In a second phase, interviews were done and key information and data were identified as evidence of the performing sub-practices corresponding to maturity level 2 and level 3. Four interviews were carried out in order to know the perception of experts and checking project documentation. In general, the perception of experts were validated by evidence; in a few cases, the perception of an expert was different. In those cases, after seeing the evidence the expert changed its evaluation.
RESULTS
The study was developed in a company which operates since 2006 and services clients from the Andean region and United States. It offers solutions for CRM (Customer Relationship Management), BMS (Business Management Systems), and ecommerce (shopping cart, digital certificate, and merchant account).
The company has offices in the United States, where it was created, and in Colombia (Cali, Bogotá and Medellín) but the project was developed in Cali. According to evidence found, the company applies 77% of the components of the model while it is missing the 23% of it. An expected result considering that company is filial of a United States based company, so best practices should be copied. However a detailed analysis found maturity level is not as high as understood with first result.
Taking into account that project management maturity model, for this project, is established from level 2 to level 4, final evaluation indicates the maturity for the company does not reach that first level of the scale as established by CMMI-DEV. Figure 1 shows how the level 2 is almost reached while level 3 and level 4 are in lower performance.
As shown in figure 1, level 2 is almost reached. Requirements management (REQM) is fully implemented while supplier agreement management (SAM) and project monitoring and control (PMC) are almost fully implemented by registering a score of 90%, followed by project planning (PP) that registered 89% of implementation.
Reaching maturity level 3 is harder for the company because, when calculating the average of processes, it has implemented only 52% of the processes. Although integrated project management (IPM) is implemented in 87%, the other process, risk management (RSKM) is only implemented at 17%. A paradox found in the research is that the company has higher implementation in quantitative project management (QPM), 64% implemented, a process required for level 4, while it still has a level 3 process with only 17% of implementation.
In another analysis level, it could be found how goals have been implemented for every process area. At the goals level, 15 goals were evaluated. It has to be noticed that the number of specific goals for a process is variable.
The 53% of total goals are partially met and the rest were not met. As the instrument follows a hierarchical structure, the evidence shows consistent assessment as obtained for process areas. As can be seen in table 5, Goals related with RSKM process are not met while in QPM process the weakness related with one objective is leveled with a partially met objective.
Finally, out of the 59 practices evaluated, 25 are fully implemented which means that are carried out. As seen in figure 2, the remaining 23 are largely implemented and 7 are partially implemented. And finally, 4 are not implemented which means that all sub-practices are missing. Data shows consistency with previous assessment, if consider that 9 from the 11 practices evaluated as partially implemented and not implemented are located in RSKM and QPM processes. The other 2 practices are located in PP and PMC processes but the higher number of practices fully implemented covers that weakness.
From a different approach, not considered by CMMI, practices and sub-practices were classified in planning, executing, and controlling basic phases. Some practices were classified in different phases at the same time because sub-practices permitted that classification as seen in Table 6.
By using this approach, similar patterns were identified from fully implemented to not implement. For executing processes, a slightly different pattern was identified, where fully implemented sub-practices are lower than largely implemented subpractices as shown in Figure 3.
CONCLUSIONS
A project management maturity model is an excellent tool to identify strengths and weaknesses in a company, however the structure and the way the evaluation is performed need to be defined in every case. In that sense, the expertise of the analysts and experts in the company is fundamental.
An important first step is the selection of the maturity model. For this research, the scope was to work in the information technology industry, so the selection of the model was a specific task. However for companies from other industries a specific model may not be available so a general model should be modified and used.
Components from the model such as satisfaction scales, multiple criteria for experts' selection and redundant evidence evaluation, allow it to obtain reliable information about the maturity level. The quantitative evaluation scale and the color code designed resulted in being a good support for the analysts and the experts when collecting information and later when analyzing it.
The redundant evaluation, the expert views and evidence found were helpful to solve a few questions, but it could be definitive in cases where perceptions of experts result very different. Project management maturity assessments usually depend largely on expert perception. Although the selection of experts could provide some confidence in results, it has to be remembered that this kind of assessment is perception-based. It should be improved by validations from analysts. Criteria for selecting experts included a filter to decide who could take part in the research. However the question of how many to select, or what to do if none matching the filter showed up.
The evaluation showed that the company does not reach the maturity level 2 in project management, since not all areas of process are satisfied. However, there is a high coverage of 92% in the application of the CMMI model components in this level, which means that carrying out an improvement plan that offsets the weaknesses found, it could reach maturity level 2.
The lowest percentage of coverage in processes areas was obtained in Risk Management and Quantitative Management of the project, with 17% and 64% respectively, which belongs to the level 3 and level 4. If the company is considering an improvement plan in these areas, it must be aware that it is needed to improve initially the level 2 process areas. The CMMI model proposes that reaching a maturity level is only possible when the previous level has been reached.
Future research, in order to validate and improve the proposal could consider validation in a group of companies that could provide different situations such as different processes, size of the company, and more experts.
ACKNOWLEDGMENTS
The authors thanks to Universidad del Valle for supporting the field research and the company which accepted to participate in the research.
REFERENCES
Aguirre, S., & Córdoba, N. B. (2008). Diagnóstico de la madurez de los procesos en empresas medianas colombianas. Ingeniería Y Universidad, 12(2), 245–267.
Ahmed, F., & Capretz, L. F. (2011). A business maturity model of software product line engineering. Information Systems Frontiers, 13(4), 543–560.
Allué, A., Domínguez, E., López, A., & Zapata, M. a. (2013). QRP: A CMMI Appraisal Tool for Project Quality Management. Procedia Technology, 9, 664–669. DOI: 10.1016/j.protcy.2013.12.073
Arce, S., & López, H. (2010). Valoración de la gestión de proyectos en empresas de Bogotá. Revista EAN, (69), 60–87.
Benavides, J., Castro, F., Devis, L., & Olivera, M. (2011). Impacto de las Tecnologías de la Información y las Comunicaciones (TIC) en el Desarrollo y la Competitividad del País, 225.
Chevers, D. a. (2014). Key Factors of Process Maturity in English-Speaking Caribbean Firms. Factores Clave de La Madurez Procesal En Las Empresas de Habla Inglesa Del Caribe., 54(3), 322–329. DOI: 10.1590/S0034-759020140307
CMMI Institute. (2011). Standard CMMI ® Appraisal Method for Process Improvement ( SCAMPI SM ) A , Version 1 . 3 : Method Definition Document. http://doi.org/CMU/SEI-2011-HB-001
CMMI Institute. (2013). Standard CMMI ® Appraisal Method for Process Improvement ( SCAMPI) Version 1.3a: Method Definition Document for SCAMPI A , B , and C. CMMI Institute.
Coleman, G., & O'Connor, R. (2008). Investigating software process in practice: A grounded theory perspective. Journal of Systems and Software, 81(5), 772–784. DOI: 10.1016/j.jss.2007.07.027
Ehsan, N., Perwaiz, A., Arif, J., Mirza, E., & Ishaque, A. (2010). CMMI / SPICE based process improvement. Management of Innovation and Technology (ICMIT), 2010 IEEE International Conference on, 859–862. DOI: 10.1109/ICMIT.2010.5492803
Ensslin, L., Scheid, L. C. M., Ensslin, S. R., & Lacerda, R. T. de O. (2012). Software Process Assessment and Improvement Using Multicriteria Decision Aiding - Constructivist. Journal of Information Systems and Technology Management, 9(3). 473-496. DOI: 10.4301/S1807-17752012000300003
Garzás, J., Pino, F. J., Piattini, M., & Fernández, C. M. (2013). A maturity model for the Spanish software industry based on ISO standards. Computer Standards & Interfaces, 35(6), 616–628. DOI: 10.1016/j.csi.2013.04.002
Iyidogan, S. (2014). Exploring the Diffusion of Software Quality Standards: Evidence from the Case of Turkey. Procedia - Social and Behavioral Sciences, 122, 362–366. DOI: 10.1016/j.sbspro.2014.01.1355
Jamaluddin, R., Chin, C. M. M., & Lee, C. W. (2010). Understanding the requirements for project management maturity models: Awareness of the ICT industry in Malaysia. IEEM2010 - IEEE International Conference on Industrial Engineering and Engineering Management, 1573–1577.DOI: 10.1109/IEEM.2010.5674174
Jiang, J. J., Klein, G., Hwang, H.-G., Huang, J., & Hung, S.-Y. (2004). An exploration of the relationship between software development process maturity and project performance. Information & Management, 41(3), 279–288. DOI: 10.1016/S0378-7206(03)00052-1
Mesquida, A. L., Mas, A., Amengual, E., & Calvo-Manzano, J. a. (2012). IT Service Management Process Improvement based on ISO/IEC 15504: A systematic review. Information and Software Technology, 54(3), 239–247.DOI: 10.1016/j.infsof.2011.11.002
Ministerio de Comercio Industria y Turismo. (2013). Promotional Strategies and Policies for Developing Local Economy and Establishing Science and Technology Parks in Colombia.
Ministerio de Tecnologías de la Información y las Comunicaciones. (2012). Informe sectorial de la industrial de software y servicios asociados de Colombia.
Ministerio de Tecnologías de la Información y las Comunicaciones. (2013a). Plan Vive Digital. Dirección de Políticas y Desarrollo de T.I. Plan Vive Digital.
Ministerio de Tecnologías de la Información y las Comunicaciones. (2013b). Visión estratégica del sector de software y servicios asociados plan de mercadeo y ventas regionalizado del sector en colombia. Plan Vive Digital.
Pérez Mergarejo, E., Pérez-Vergara, I., & Rodriguez-Ruíz, Y. (2014). Modelos de madurez y su idoneidad para aplicar en pequeñas y medianas empresas, XXXV(2), 146–158.
Selleri, F., Santana, F., Soares, F. F. S. F., Lima, A., Monteiro, I., Azevedo, D., … Meira, S. R. de L. (2015). Using CMMI together with agile software development: A systematic review. Information and Software Technology, 58, 20–43. DOi: 10.1016/j.infsof.2014.09.012
Shih, C.-C., & Huang, S.-J. (2010). Exploring the relationship between organizational culture and software process improvement deployment. Information & Management, 47(5-6), 271–281.DOI: 10.1016/j.im.2010.06.001
The Standish Group International. (2013). CHAOS MANIFESTO 2013. The Standish Group International.
Valencia, L. S., Villa, P. A., & Ocampo, C. A. (2009). Modelo de calidad de software. Scientia et Technica Año XV, (42), 172–176.
Villalba, R. L., & Estrada, L. (2011). Aprendizaje y aplicación del modelo CMMI-DEV en pymes de software Colombianas-La experiencia RCCS. Gerencia Tecnológica Informática, 9, 57–76.
Von Wangenheim, C. G., Silva, D. A. da, Buglione, L., Scheidt, R., Prikladnicki, R. (2010). Best practice fusion of MMI-DEV v1.2 (PP, PMC, SAM) and PMBOK 2008. Information and Software Technology, 52(7), 749–757. DOI: 10.1016/j.infsof.2010.03.008