AN EXTENDED ECOLOGICAL POINT OF VIEW TO EVALUATE TECHNOLOGICAL INNOVATIONS
Giovanni Moura de Holanda [1]
Universidade Estadual de Campinas, UNICAMP
gmholanda@yahoo.com.br
Carolina Vaghetti Mattos [2]
Centro de Tecnologia da Informação Renato Archer - CTI
carolina.mattos@cti.gov.br
Angela Maria Alves [3]
Centro de Tecnologia da Informação Renato Archer - CTI
angela.alves@cti.gov.br
______________________________
Abstract
This paper describes an analytical approach based on a non-conventional ecological perspective to evaluate the quality of technological innovation projects. The approach is semi-qualitative and focuses on aspects of cooperation in projects and companies with favorable conditions regarding technological advancement and learning maturity, highlighting contributions of these aspects to the sustainability of R&D projects. Moreover, it considers the multidimensionality of a project in order to classify how its innovations align with sustainable principles, in terms of balancing the needs of all stakeholders. Beyond technology, the multiple dimensions associated to a project and its impacts comprise, for example, the economic, social, human and environmental spheres; similarly to the arrangement of forces for survival and evolution in comprehensive ecosystems, hence the allusive use of the term “extended ecological perspective”. Complexity theory and interrelation of the concepts of sustainability, cooperation, learning and multidimensionality underlie this analytical approach of innovations resulting from R&D efforts.
Keywords: Project evaluation. Cooperation. Innovation. Sustainable development. Research policy.
UM PONTO DE VISTA ECOLÓGICO ESTENDIDO PARA AVALIAR INOVAÇÕES TECNOLÓGICAS
Resumo
Este artigo descreve uma abordagem baseada em uma perspectiva ecológica não convencional para avaliar a qualidade de projetos de inovação tecnológica. A abordagem é semi-qualitativa e focaliza os aspectos de cooperação em projetos e empresas com condições favoráveis em termos de avanço tecnológico e maturidade de aprendizagem, destacando contribuições desses aspectos para a sustentabilidade de projetos de P&D. Ademais, considera a multidimensionalidade de um projeto no sentido de classificar como suas inovações alinham-se com princípios de sustentabilidade, no que se refere ao equilíbrio das necessidades de todos os stakeholders. Além da tecnologia, as múltiplas dimensões associadas a um projeto e seus impactos compreendem, por exemplo, as esferas econômica, social, humana e ambiental; similarmente ao arranjo de forças para a sobrevivência e evolução em ecossistemas abrangentes, daí a alusão ao termo “perspectiva ecológica estendida”. A presente abordagem analítica das inovações resultantes dos esforços de P&D é orientada pela teoria da complexidade e pela inter-relação dos conceitos de sustentabilidade, cooperação, aprendizagem e multidimensionalidade.
Palavras-chave: Avaliação de projeto. Cooperação. Inovação. Desenvolvimento sustentável. Política de pesquisa.
1 Introduction
Companies that maintain Research, Development and Innovation (RD&I) activities deal to some extent with the practical and conceptual aspects of cooperation. Expanding technical and market domains of action, seeking innovations in products and processes, developing technological capacity building and establishing competitive differential necessarily involve collaborative work.
In organizational terms, there are several indications that in dynamic markets and R&D environments, cooperation and collaborative practices foster innovation, value creation and performance of firms (BEERS; ZAND, 2012; FAEMS et al., 2005; BELBERDOS et al., 2004). Companies often unite to share goals, add efforts, reduce costs, renew strategies and maintain competitive advantages. Some steps have already been taken in this direction, but there is still a field of action in which is it possible to observe a certain taboo, whose barriers have not been removed and where cooperation can be even more intense.
It is common for some companies to think about how much they can associate with each other without revealing details of their strategic plans and how to disseminate innovations among partners without compromising their competitive advantages. Part of this apprehension is due more to a limited understanding of the concepts of collaboration and cooperation, and to a feeling of market share preservation, rather than concrete threats of a competitive environment.
Learning, capacity building and technological innovation are strongly interrelated in the sphere of RD&I efforts. By revising theoretical publications and empirical findings on this matter, we strengthen our position that cooperation and collaborative research may lead companies to remain not only competitive, but also to consolidate themselves in the market in which they operate, with regard to the achievement of organizational objectives and the sustainable development.
Several studies have addressed the prolific relation between cooperation, innovation and R&D efforts – for example, from a European perspective (EUROPEAN COMMISSION, 2016; BEERS; ZAND, 2014; TETHER, 2002), based on American firms’ strategies (CHESBROUGH, 2012) and on Chinese firms (WU, 2014), and focusing on emergent economies, specifically trends in Brazil and Argentina, (BUENO; BALESTRIN, 2012; EDWARDS-SCHACHTER et al., 2011; KUPFER; AVELLAR, 2009). Assessing how much companies and projects can benefit from these cooperative efforts and attitudes is important in order to provide parameters of organizational performance and provide means for all stakeholders to benefit from synergy and share actions.
The involvement of a broad range of actors in the innovation process has been considered a key issue. For example, in the OECD (2015, p. 6) innovation strategy such an involvement appears as one of the main features of innovation as seen today, and the range of actors includes “firms, entrepreneurs, foundations and non-profit organizations, universities, scientific institutes, public sector agencies, citizens and consumers, often working in close collaboration”.
As projects reach a certain degree of maturity, both in terms of their systematic processes of development and the technological advances made possible by such processes, it becomes necessary to have a more detailed view of the very evolutionary R&D line that a company is following. This need is indicated in a recent work (HOLANDA et al., 2017), as an opportunity for deepening the conceptual and methodological frameworks of assessment of R&D projects. In that paper, there is a suggestion to expand the understanding of “competitiveness” by including the effects of cooperation on innovation and technological capacity building. Such an expansion was also inspired in Alves (2013), when she pointed out that competitiveness should include the concepts of "cooperation" and "sharing" in order to maximize collaborative efforts and achieve common goals.
Motivated by these suggestions and seeking to visualize competitiveness as a characteristic of survival and outstanding performance in a given environment, we believe that it is plausible to extend the notion of competitiveness by associating it with the concept of ecology in a somewhat broader sense. Characteristics of this concept are particularly useful in consolidating a new paradigm to understand competitiveness from a higher perspective, for example, where the evolutionary equilibrium between all parts of a given ecosystem become an ultimate purpose, similarly to living systems.
The essential properties of living systems can be explained by complexity theory, through the dynamics and multidimensionality that characterize complex systems in a unique way. Thus, another aspect that delineates sustainable innovations, with capacity to reach the aforementioned balance and be socially and financially self-sustainable, is the multidimensionality involved in the design and evaluation of RD&I projects. While making a comparison with the notion of ecology, we are not applying these discipline principles to evaluate eco-innovations or to study aspects of green economy. The intention is to incorporate and expand the holistic paradigm inherent to the ecological perspective to analyze technologies applied to any kind of social and marketing ecosystem. In doing so, we also visualize technologies as having in the innovation process their own autopoiesis, in the sense settled by Maturana and Varela (1973), with ramifications in all domains of reality.
In this sense, the present paper brings an approach based on an extended ecological point of view to evaluate the quality of technological innovation projects in terms of learning and innovation. The integrative approach focuses on aspects of cooperation in projects and firms that have achieved favorable conditions in terms of technological advancement and maturity in their development processes (consolidated as learning), highlighting the degree of contribution of these aspects to the sustainability of projects. Moreover, it considers the multidimensionality involved in the project in order to classify how its innovations are aligned with sustainable principles. The approximation of the concepts of sustainability, cooperation, learning and multidimensionality underlies this analytical perspective of innovations resulting from RD&I projects.
2 Some underlying concepts
Sustainability and, a fortiori, sustainable innovations have not unique nor even consensual definitions, mainly because they are terms within an extensive discussion. Sustainable development was defined in Brundtland Report (UNITED NATIONS, 1987, p. 41), as the “development that meets the needs of the present without compromising the ability of future generations to meet their own needs”. Deepening such a concept, the report states that the efforts able to promote sustainable development must all be harmonious in order meet human needs and aspirations (UNITED NATIONS, 1987, p. 43). This seminal definition has nurtured and shaped several other definitions since then (UNITED NATIONS, 2015).
With the United Nations Conference on Environment and Development, held in 1992 in Rio de Janeiro, the conceptualization of sustainable development integrates three pillars: economic development; social development; and environmental protection (STODDART, 2011). In turn, the 2002 World Summit on Sustainable Development represented another expansion on the Brundtland definition by highlighting the interdependence and reciprocal influence of these three pillars (KATES et al., 2005).
One can note that this conceptualization is related not only to individuals, organizations and environment but also to their inter-relations. Following this conceptual evolution, we share the same understanding of Pastoors et al. for the term sustainability, which is related “not only for every individual and every company, but also for humanity as a whole” (PASTOORS, 2017, p.9).
Coherently, we assume “sustainable innovations” as being aligned with complexity theory, involving therefore multiples dimensions. These dimensions are, for instance, social, economic, cultural, political, organization and human. In a word, sustainable innovations should provide social value with minimal consumption of resources and without jeopardizing any aspect of cultural or cognitive nature, in such a way we consider sustainability as being structured by at least three interrelated dimensions: the social, economic and environmental.
An extended ecological point of view corresponds in this digression to an integrative approach, which includes simultaneously as many dimensions as possible. Therefore, it is employed here in a sense of a more comprehensive analytical approach, with potential to enhance the vision about the interrelations that characterize the efforts to survive (or be competitive) in an ecosystem (biological or technological). Analytical approaches centered on the ecological vision differ from the strict environmental approach, which in turn comprises just a few dimensions, tends to isolate the parts of an ecosystem, and is predominantly driven by a classical economic perspective. Even if the detachment between object and subject may be still observed, a comprehensive and multidimensional approach – or an extended-ecological point of view, as we are proposing to name it – narrows this distance, reinstate the dimensions and seek to contextualize humans, technologies and nature in the same habitat.
Ecology and technologies share dynamic properties, are essentially complex, require multi and interdisciplinarity, exhibit diversity and information circularity. This metaphoric comparison is both a practical and philosophical way to articulate theories, concepts and even non-conventional methods to analyze complex systems, such as RD&I projects.
In this context, technology is considered an important vector of changes in all spheres of human action, creating new patterns of life, social organization, productive forces and resizing the relation with the environment. Consequently, new technologies and even innovative uses of current ones may be applied for promoting sustainability.
Cooperation and collaboration come from the same essence of mutual and shared action, in which the common goal is to combine efforts, reduce costs and work energy, and produce results more favorable to the objectives of organizations involved in joint ventures. Concerning the aims of each institution within collaborative contexts, the ethos and organizational culture, the form of participation in joint actions vary, as well as the benefits that can be achieved. The competitiveness that is usually sought, even in collaborative and cooperative business environments, can be specified in different scopes and contours.
Defining and establishing boundaries between "collaboration" and "cooperation" is semantically complex, by either etymological aspects or practical use. The two terms are often marked by ambiguities or by their interchangeable use, usually treated as synonyms. Several authors have dedicated themselves to this semantic and conceptual challenge (LAI, 2011; HARLEY; BLISMAS, 2010; FRANCO; BARBEIRA, 2009).
Some degree of cooperation has always been present in organizations (FRANCO; BARBEIRA, 2009). Among the variations between collaboration and cooperation indicated in the survey carried out by Harley and Blismas (2010), we opted for the understanding that “cooperation” corresponds to an approximation and a more intense commitment between the companies participating in a joint effort, if compared with the term “collaboration”. By the way, this semantic choice is even contrary to that adopted in (HARLEY; BLISMAS, 2010). The rationale that has led us to such discernment is provided above all by the work of Piaget.
In Piaget's view, the method of cooperation can be compared to a scientific attitude, in which an individual need to decenter from their point of view to know other perspectives – see (CAMARGO; BECKER, 2012). Working together without the notion of the whole, the complete vision of intended objectives, is just collaboration. Cooperating means working or operating entirely together. Collaborating is just contributing to the work.
In the study presented here, we use the term "cooperation" as a more intense and lasting relationship than that of a "collaboration". The former denotes the situation in which two or more organizations use each other’s resources and expertise in a fully committed way, aiming at a common mission to achieve the same objectives and targets, set out in a joint work plan.
In the conceptual framework of our approach, detailed further ahead, collaboration precedes cooperation, and both complement each other in order to cumulatively behave as a vector for learning and innovation, which we named “cooperativeness”.
3 Cooperativeness as vector of learning and innovation
A company needs to assimilate flows of information and knowledge in order to perform appropriately, which requires some degree of learning. Projects that lead to a high learning rate are usually associated with significant innovations in both the developed products and the R&D processes that have been developed and implemented.
A systemic model of innovation takes into account that innovation occurs within a networks of direct and indirect relations, with cooperation as a basic catalyst. Torres et al. (2004) reflect on the relationship between learning and cooperation, presenting an approach to classify the level of interaction that characterize a process of cooperation and thus assessing the intensity of this cooperation between actors of a local productive arrangement.
The current strategic policies are very different from those practiced a few decades ago. The accumulation of company capacity involved in R&D relates directly to the speed of the technological frontier. According to Miranda and Figueiredo (2010), a consequence of this factor is that a firm may not reach that frontier if its capacity building speed is lower than the technological frontier speed.
In this fast and dynamic environment, cooperation between companies tends to be a factor of equilibrium in learning rates and an inducer of innovation processes. In general, the process of technological learning can occur within the company or in the external relationship, based on interactions with suppliers, partners, consumers and with the scientific and technological structure, both locally and regionally. The mode through which learning takes place and the acquired knowledge is applied is determinant for the internal processes of the company and for the emergence of innovation opportunities.
In fact, with the technology acceleration and increasing market complexity, cooperativeness between firms or within innovations networks became an important arrangement to mobilize adequate knowledge in order to generate innovations and foster technological change. These external relationships characterize a way of cooperation that is needed in order to keep pace with the current demand for new and more intensive practices of innovation. Such a dynamic and complex mesh of interrelations is better described as a non-linear model of innovation.
There are many approaches and theoretical and empirical studies on the cooperation potential of innovation networks. In the mid-nineties, Freeman (1995), for example, presented an historical perspective on the importance of national systems of innovation. Fischer et al. (2001) highlight the feedback and interactions in a model for the innovation process. Morrar et al. (2012) focus on strategies for service innovation addressing cooperation between public and private sectors.
Exchange and relationships are therefore at the basis of the construction of knowledge and processes of innovation outlining contemporary economies, especially emerging ones that need to deal with even more specific difficulties. When discussing indicators of innovation and cognitive capitalism, Giuseppe Cocco (2010), emphasizes that the value creation of knowledge can no longer be observed within the company, but in the field of cognitive networks and their interorganizational relationships.
In this context, knowledge, learning and technological capacity-building, on the one hand, and innovation on the other, influence each other and have collaboration and cooperation as factors of expression. Figure 1 brings a depiction of that process of interdependence, from the perspective of the orthogonal relationship between learning and innovation. Orthogonality in this case is pure pictorial contingency of a bilateral relation, without representing these two factors as being of exclusive or independent nature. This representation intend to place the cumulative effects of cooperation on the development of learning and simultaneously on the driving forces to innovation. Cooperation represents the maximum degree of collaborative efforts and joint work in learning processes.
|
Figure 1 - Factors conditioning learning, innovation and sustainability. |
|
|
|
Source: The authors, 2021. |
In the representation suggested by Figure 1, a project whose principal characteristics associate it with "coopetition" (a semantic hybrid formed by the terms collaboration and competition) translates into a condition of innovation and competitive differential achieved by applying collaborative learning processes. Coopetition, in its various degree resulting from corporate alliances, has aroused the interest of academics and has been addressed in many studies over the last twenty years, focusing on its effects over innovations and firm performances. By way of illustration only, Wu (2014) and Luo et al. (2007), present studies on cooperation between competitors and rivals, whereas Tsai (2002), addresses coopetition in a multiunit organization.
The plethora of studies highlights that cooperation and alliances with competitors may strengthen or weaken the expected performance in terms of innovation, financial and market positions, depending obviously on the strategy adopted by companies. The fusion of cooperation and competition is nevertheless a paradoxical state that, like complex adaptive systems, is guided by the contradictory dynamics between these two actions in a stage of creative tension. Here again we find similarity with the concept of living systems, reinforcing our use of the ecological metaphor, employed within the current approach for evaluating R&D projects.
From an operational perspective, organizations may be considered as complex adaptive systems – see, for example, (STACEY, 1996). Since such systems are ever transforming in order to maintain a state of equilibrium and evolve, we can understand that such transformation is also at the basis of organizational learning. In accordance to Sterman (2000), the fact that systems are adaptive is one of the reasons that contribute to dynamic complexity. Adaptation also occurs as people learn from experience, when they seek new ways to achieve their goals and overcome obstacles.
4 Multidimensionality and cooperativeness as credential for sustainable innovations
Just as learning and innovation are inter-related, multidimensionality expressing the complexity of a project reflects to some extent the degree of innovation achieved, indicating how comprehensive was its design and how holistic is the evaluation process of its impacts. Innovation is considered, in our approach, as an emergent sociotechnical phenomenon resulting from complex interactions between different agents and requiring an integral understanding of the properties of the ecosystem in/to which the project is conceived and developed. Increasing perception of the complexity of such phenomena has highlighted the greater relevance of multi and interdisciplinary approach to ecosystem nuances.
A number of approaches seek to address the technological, economic, and social complexities of innovations. Using Morin’s “complex thought” to address the relation between technology and complexity, Malaina (2009), poses a multidimensional approach to think technology in a complex social system. Matei and Antoine (2015), apply complexity theory to analyze the links between innovation and adaptive social systems in the context of innovations developed to fulfill social needs. On the other hand, the economic dimension of innovation is largely dealt with in a specific handbook by Antonelli (2011).
Nevertheless, due to the inherent complexity of R&D projects and technological innovations, social, economic, and technological dimensions should be considered together, circumscribed by a unique ecosystem (or meta-ecosystem) with multidimensional properties. Inasmuch as more dimensions are taken into account more chances the project has to be sustainable, since the perspectives of all stakeholders may be considered. If the R&D project has requisites of economic, social, human and environmental order, beyond the technological component, more needs can be met. The same concerns of multiple dimensions are valid for evaluating project performance and impacts, which can provide valorous feedback for correcting faults, minoring risks, preventing pitfalls and improving quality. In general, such analytical and design orientations contribute to more sustainable long-term results.
Returning to the representation of Figure 1, a project with a high degree of learning and a high level of innovation presumably reaches a condition of sustainability, in which the intensity of cooperation is the reflection of a fair balance among all the interests involved, and innovation is the solution that best responds to the needs set to be met. In addition, the results of cooperation fostering learning may be perceived in terms of experience interchange, acquired skills, and capacity building.
It can be said that projects with these characteristics of learning and innovation are sustainable and reached a stage of solidity of purpose that allows linking another term to their qualification: solidarity, in the sense of solid, which sustains itself. By acting in solidarity, a cooperating person or organization works for the construction of something collective, for the "(con)solidation" of common goals.
At this stage of sustainability, companies have already transcended the Schumpeterian notion of competitiveness, specifically the “creative destruction". It is clear that survival in the market and economic performance are aspects of paramount importance for companies. However, in Figure 1, sustainability involves other characteristics, which correspond to a more comprehensive pattern of consciousness. In such a condition, for example, companies are not seeking the recurrent destruction of the old to create the new as a strategy to continuously generate demand or just to be a market pioneer.
Moreover, in Figure 1, the relationship between innovation and sustainability does not refer to the sense found in Christensen's theory (2003), in which “sustaining” innovations are understood as improvements to make products more competitive, counteracting disruptive innovations that bring totally new products to market. In the present approach, sustainability is achieved by innovations not only aimed at providing better products in relation to previous versions and keeping them competitive, but by innovations whose benefits and impacts are perceived in a broader sense of ecology that we are trying to incorporate. Such innovations aim not only at maintaining the market position already achieved, but at leading to sustainable development, associated with long-term results and impacts, with fair and balanced use of production and consumption resources, and meeting the needs of current and future stakeholders. Naturally, such an orientation necessarily includes choices and care to preserve the environment.
From the moment instrumental rationality no longer responds to the inherent complexity of sociotechnical systems typical of modernity, ecology gains space by trying to connect nature and culture, environment and the production of knowledge, the human and the nonhuman – the latter binary following Latour’s dualistic view about technology hybridism (LATOUR, 1991). The challenges posed by the market to the industry of products and services, the role of innovation as a lever for development and new scientific paradigms highlight this notion of ecology and sustainable solutions, in which the interests of all stakeholders (industry, investors, government, society, individuals, and the environment) must be met in the short, medium and long term[4].
As work results become more immaterial and the means of production are more dependent on the ever-expanding knowledge base and on the interaction between individuals and organizations, the aspects of learning and cooperation become essential and catalyst of technological and productive advance. The immateriality of labor, expressed in the exchange and propagation of knowledge, is much richer in meaning and value than labor in the production of material goods – such trend is more noticeable in Information and Communication Technologies (COCCO, 2010).
Evaluating such values in a collaborative work is unusual, however, and mainstream methodologies tend to prioritize quantitative approaches and data analytics. On the other hand, in his contribution to a theory of qualitative values, Piaget (1973), many decades ago, already expressed some rules of correspondence within a scale of social values of exchange. By modeling this exchange, he assumes the following nomenclature (PIAGET, 1973):
a) ra corresponds to the action (or reaction) of a over a’;
b) sa’ is the satisfaction of a’ provided by the action ra; and
c) va represents the valuation of a by a’.
Thus, for an exchange of values in a cooperative environment, both interindividual and collective, it is possible to establish a relation of values. For example, when there is mutual benefit or even if the production efforts are overvalued by a’, due to results that exceeded expectations, it is possible to express:
(ra < sa’) and (va > ra);
Similarly, when there is balance of actions and valuation:
(ra = sa’) and (va = ra).
The same logic can be applied to situations in which there is devaluation or imbalance. What is important in this theorization is not to seek laws of equilibrium of economic exchanges, of strictly quantitative nature, nor specific rules for exchange of qualitative values, but above all to lay the foundation of paradigms that allow some kind of measurement of cooperation efforts confronted with the benefits accrued from the cooperation between organizations. In this manner, it is useful to bring qualitative approaches to work together with the quantitative methods in analyzing the impacts of cooperation processes (cooperativeness) in learning and innovation of R&D projects. Semi-qualitative approaches may be especially adequate to deal with complexity and its unexpected effects, besides behaving as complementary method to statistical analyses.
5 An alternative for categorizing RD&I projects
In contrast with the modus operandi of companies acting in isolation, cooperation between organizations can bring a number of benefits to RD&I projects and to all stakeholders. Cooperation makes possible, for example, to combine skills and share savoir faire; add effort and prorate R&D costs; minimize innovation risks by using knowledge from various bases of experience; reduce time of new developments; increase the quality of products by the exchange of experience between research centers, companies, suppliers and consumers; and expand markets by the joint action and removal of entry barriers.
In an increasingly competitive scenario, companies have decided for cooperation or strategic alliances. As argued by Sierra (1995), some factors favor this, among them competition through technology and R&D, joint efforts to increase competitiveness and entry into markets that are already dominated, and competition by means of integrating technologies and markets, due to the complexity and costs involved in various technological fields.
Networks of companies have been adopted in this context. By acting within a network, companies can complement each other by establishing interdependencies in technical, productive, research and marketing terms. The formation of clusters is already widespread today, when the sectoral and geographic aspects are concentrated for the establishment of technology parks and the increase of collective efficiency.
As we have argued so far, cooperativeness may be considered as a vector that propel learning and technological innovation. Following an evolutionary line, one can attribute the intensification of collaboration and cooperation processes to a condition of sustainability. It is possible to establish a positive correlation between collaboration and cooperation, constituted learning and the degree of innovation of a project. The more elements of cooperation employed in a project, the more chance of achieving long-term sustainable results. In the same way, innovation is directly related to sustainable solutions, according to the ecological paradigm mentioned previously.
The ethos characterizing such condition and paradigm is strongly conditioned by the teleology underlying the project. Projects in this condition are motivated not only by their scientific and technological potential, but also by their applicability and social value, including in this sphere the optimization of the use of resources. It is precisely this augmented vision of values that lifts up a project to the level of sustainability. In a sense, the Pasteur’s quadrant approach, proposed in the nineties by Stokes (1997), was already concerned with the social value of a project, classifying in this category researches that seek new knowledge and are simultaneously inspired by use.
Such a correlation of factors[5] conditioning innovation and sustainability, illustrated in Figure 2, can guide the identification and proposal of appropriate indicators to evaluate R&D projects that are at a certain level of maturity and cooperation.
Certainly, when we refer to the maturity of a project, previous quality levels should be considered. For example, including criteria to identify minimum degrees of innovation and capacity building that classify a project as being, in fact, a R&D project – see (HOLANDA et al., 2017). Once these levels have been reached, the project is qualified for the quadrant analysis, focused on cooperation practices and the complexity involved, as discussed in the following paragraphs.
|
Figure 2 - Vectors for sustainability of R&D projects. |
|
|
|
Source: The authors, 2021. |
Cooperativeness, expressed by cooperative practices, and project quality have a direct relationship. For instance, by proposing a reference model for the public ecosystem of software production, Alves (2013), concludes that this model allows evaluating the maturity of digital ecosystems with emphasis on cooperation and sharing, differently from conventional models. As in a natural ecosystem, the resulting dynamics is expressed by means of interaction, sharing and learning, which are characteristics of an ecosystem (ALVES, 2013).
In addition, another vector that favors the quality of a project, positively expressed in this case by the "sustainability" aspect, is related to the number of dimensions considered in the project or policy. In dynamic contexts that characterize the current scenario, it is fundamental to consider projects and policies according to the social, economic, technological, political, cultural and human dimensions. The more they are treated as a complex and multidimensional system, shaped by the perspectives of all those involved, the greater the likelihood that R&D efforts will be successful and that sustainable solutions will be obtained.
Table 1 can be suggestively used as a road map for the specification of a metrics system that considers the impacts of cooperativeness on the relationship between learning and innovation. Of course, applying said metrics system is intended as a mode of evaluation for projects that have showed, in previous analyzes, evidence of innovation and technological capacity building.
Examples of what can guide and configure a metrics system based on this approach are:
d) Type of alliance between the participating companies;
e) Number of companies involved;
f) Existence of an agreement with local and regional IST (Institutes for Science and Technology);
g) The way results are shared;
h) Participation in network of companies;
i) The way of propagation and use of the constructed knowledge;
j) Methodology of research;
k) The interdisciplinary nature of the collaborative work;
l) Participation in local clusters; and the set of parameters for sustainability.
On the latter, some systems of sustainability indicators are being developed worldwide – an overview of such systems is presented, for example, in (BELLEN, 2006).
|
Table 1 - Quality aspects of R&D projects: road map to indicators. |
||||||||||||||
|
||||||||||||||
|
Source: The authors, 2021. |
The dimensional component, with respect to the methodological approach contemplated by the project and the collaborative efforts employed, is also a reflection of the quality of R&D projects. When considering a project or sectorial policy as being multidimensional, with requirements and impacts in all spheres of a sociotechnical system – that is, economic, political, regulatory, cultural, human and naturally social and technological – the chances of conforming the project to the reality and needs of an entire network of agents increase significantly.
Still on this methodological aspect, it is equally relevant to employ interdisciplinary techniques and methods for the appropriate treatment of the complexity of sociotechnical systems. Moreover, it is crucial to consider the perspectives of all stakeholders throughout the product life cycle, while planning requirements and functionalities, as well as while evaluating the impacts verified in its implementation and operation.
Similarly to cooperativeness, the methodological resources and the multidimensional vision of a project contribute to its quality regarding innovation and learning. Both vectors strengthen the learning process, either at the level of individuals or participating organizations, and behave as catalysts for innovation. Thus, innovative projects and policies, which are strongly driven by these factors, provide intense learning and, by extension, should present considerable characteristics of sustainability, taking into account multidimensional impacts and the needs of all stakeholders, including the environment.
An evaluation process based on a metrics system as suggested in Table 1 can indicate the quality of a R&D project in terms of learning and innovation. The positioning in the quadrants formed by the relation between the learning and innovation axes – see Figure 1 for a spatial visualization – stems from the attribution of weighted values to the quality indicators provided by the metrics system. The numerical formulation expressed in (1) serves as reference in this sense, indicating the coordinates of the x and y axes, in which the project is evaluated.
|
|
In this numerical scheme, pi and pj represent the weighting factors for the indicators associated with the learning (Qx) and innovation (Qy) axes respectively. Thus, the metrics weighted indicate the position of the evaluated project in the quadrant, so that a project reaches its optimum point of R&D quality when it is positioned in the upper right corner. In such a situation, the project (or policy) presents high levels of learning and innovation, and consequently achieves a high degree of sustainability. The calculated values for the parameters associated to cooperation and multidimensionality determine the project position in the quality evaluation, according to the weighting criteria (expressed as pi and pj) established for the analysis context.
Weighting criteria are semi-qualitative factors highly sensitive to this analysis and deserve special attention not to bias the results. As discussed in subsequent items, greater weight should naturally be assigned to indicators directly related to cooperativeness and multidimensionality, underpinning vectors of the current approach. The weighting effects should be chosen in a neutral and unique way in accordance with the purpose of the evaluation approach, reflecting the scope – whether intra-organizational, regional or international – and the maturity of the projects – whether the project already has reasonable levels of quality, as previously assessed by less-comprehensive methodologies. In addition, if the evaluation should have a comparative effect between projects, the weights must remain unchanged in all assessments to be carried out.
The choice of indicators depends of course on the system of metrics that can be used in this type of evaluation. The number of indicators and means of data collection should be determined together with the project managers or policy makers, as well as the weighting factors associated with each indicator.
Figure 3 shows a tree diagram illustrating the arrangement between indicators (Qx and Qy) and sub-indicators (qx and qy) in the composition of a figure of merit for the quality of project (QoP) in terms of innovation and learning. QoP corresponds to a point of coordinates, dimensioned by applying (1). With x and y coordinates, the project may be positioned in the evaluation quadrant. Note that indicators can be composed of sub-indicators, according to the granularity allowed by the metrics system.
|
Figure 3 - Generic tree of indicators for the quality of R&D Projects. |
|
|
|
Source: The authors, 2021. |
The first step towards applying this method is identifying a group of indicators and sub-indicators that can be assessed. In our first trial, which followed the road map in Table 1, we aimed at validating the conceptual approach, and selected indicators related to learning process (x-axis) and innovation (y-axis), as shown in Table 2.
|
Table 2 - A first group of project indicators. |
||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||
|
Source: The authors, 2021. |
||||||||||||||||||||||||||||||
Criteria used to select indicators are primarily adherent to available data in the project life cycle. Indicators are assessed according to four degrees. For example, investigative and experimental procedures applied (or to be applied) in the project’s execution are directly related to the learning potential. Such procedures – addressed by indicator Qx1 – mark the boundaries between R&D and related activities (OECD, 2002) and may vary from a situation with minimal and routine set of tests to homologate the development process, up to the cases in which sophisticated and exhaustive set of tests are applied for demonstrating project results. Thus, a given project has value (1) assigned to the former situation whereas another project is evaluated with maximum value (4) for the latter condition. Intermediate degrees of those aspects correspond to non-routine but non-exhaustive tests and to sophisticated and exhaustive tests but without demonstration of results, respectively.
The kind of knowledge that is object of cooperative activities, parameterized by Qx2, reflects how the project (and obviously the executing firm) is (or was effectively) inclined to open R&D activities to cooperation. This approach assumes that there is a direct and positive relation between the number of R&D activities being performed in cooperation and the firm’s learning potential. Value (1) is assigned when there is only cooperation with regard to physical infrastructure use (equipment and laboratory, for example), whereas value 4 is attributed to projects that include cooperation in core and strategic knowledge areas. Intermediate values are assigned to projects with cooperation in its peripheral areas of knowledge, or in core competence areas but without participative management.
The sort of cooperation, Qx3, reflects the extent to which cooperative arrangements are established between the projector executor and its partners in R&D efforts. The values reflect the intensity of inter-relations and the resulting learning. The lowest value assigned correspond to situations in which there is cooperation with other departments (marketing, production, other R&D units, for instance) within the company. The intermediate values are related to horizontal cooperation, i.e., with partners from the company's own value chain (suppliers, subcontractors, distributors, consumers or clients…) and ISTs (regional, national and international). The highest values reflect vertical cooperative arrangements, involving other companies (even competitors), company or innovation networks (national and international), government and impacted communities.
Indicators, linked to the learning axis, become complete with the evaluation of “mode of knowledge sharing”, Qx4, to which is attributed minimum value if project results are confined to a very small group of R&D participants, and maximum value when knowledge resulting from the project is open to anyone.
In terms of “grade of innovation”, Qy1, a project is assigned value 1 when it achieves routine results; values 2, 3 and 4 are for incremental, disruptive and radical innovation, respectively. Radical innovations are also understood as systemic innovation, in accordance to the classification shown in (OECD, 2011b). The scale to assess the scope of innovations, Qy2, obeys the logic presented in Oslo Manual (OECD, 2005), in which the lowest value corresponds to innovation at the level of the firm and the highest level implies worldwide innovation.
The values assigned to the dimensional aspect, Qy3, are directly related to the number of dimensions considered in the R&D project. As was argued earlier, the chances to achieve balance between all the interests involved increase with multidimensionality, and thus more sustainable innovations can be obtained. A techno-economically oriented project receives low grade in this criterion, while projects including other dimensions are gradually more valued, reaching maximum value when the social, environmental, cultural and human dimensions are cumulatively considered. Keeping in mind that the ecological approach adopted here is a metaphor and ecology – in its original meaning – is not dealt with in this study as equivalent to contemporary environmentalism. The environmental dimension is only one of the domains of reality considered in this analytical framework and, however, due to its contemporaneous fragility, it requires special attention in order to be integrated into the awareness of the general public.
The indicator “Multiple perspectives”, Qy4, is assigned value 1 when only one point of view is considered. For example, the company executing the project and some R&D partners. Value 1 corresponds to what we call an “ontogenic view”, whereas value 2 refers to a broader perspective, including partners in the value chain and other market agents. For the latter we have associated a “phylogenetic view”, to keep pace with the ecological metaphor, since phylogenetics regards to the relation between “species” within an ecosystem. Value 3 includes governmental and societal perspectives, and the highest value corresponds to the previous plus the environmental and end users. Resorting once more to the ecological metaphor, these last two values allude respectively to sociopolitical and ecological biomes.
It is not the purpose of this paper to provide a complete proof of concepts, but some indicators may consist of subindicators, depending on available metrics or how data is (or will be) collected. Further groups of indicators might also include information on the value attributed to cooperation between partners, i.e., how partners perceive the overall results of cooperation and the participation of each one in cooperative R&D efforts. Specific data on exchange of values in a cooperative environment may be collected in order to provide this sort of assessment, based, for example, on Piaget’s qualitative scale of values previously described. Such expansion may strengthen the analytical potential and reveal a more detailed view about the intensity of learning due to the cooperation practiced in the project execution.
Another set of indicators that may be added in subsequent versions of our approach is related to the project’s adaptability. Besides being a pinnacle of complexity theory and systems dynamics, adaptability may reveal some practices with high potential for improving the quality of a R&D project. Often innovations are responses to requirements of adaptability imposed by technological pace or prominent socioeconomic needs. Ex ante and ex post evaluation processes are related to that potential, providing important feedbacks for adjustments and corrections required by dynamic changes throughout the project’s life cycle.
The categorization of a project according to the learning and innovation aspects, leveraged by cooperative practices and the underlying methodological orientation, implies therefore an analytical approach that is consistent with the sustainable objectives expected from innovative projects in complex scenarios. Confronting obtained results with long-term collective goals and collaborative practices employed, we can examine the R&D quality achieved by projects and the extent to which organizations are committed to sustainable development.
6 A trial of two cases
Two case studies applying this extended approach are presented here. The evaluated projects exhibit a certain duality, not in terms of the Dickensian tension between two cities in a given moment of their histories, like the heading of this item might suggest (in allusion to Dickens’ well-known A tale of two cities), but by opposing quite different technology applications. That distinction regarding the type of technology and solution provided by the project characterizes an interesting scenario to assess the scope of the analytical approach.
One project, PETI-BR, is a very low cost, very easy to produce eco-innovation – more specifically a grass-root or appropriate technology –, focusing on social, human, and environmental needs. The other, CERTICS, belongs to the Information and Communication Technologies domain and is a public policy instrument for software technological innovation. The evaluation of these two projects is summarized ahead.
Case 1: PETI-BR
As described by Nascimento et al. (2016), PETI-BR is an acronym for PET Subsurface Textile Irrigation – BRazil. The project is the result of academic research and corresponds to a very low-cost subsurface textile irrigation system based on reuse of PET bottles and synthetic textile waste. The developed solution aimed both at helping rural families in drought-affected areas and at mitigating the indiscriminate disposal of PET bottles and textile waste, mainly synthetic ones. Besides being a more efficient irrigation method, it contributes to reduce the ecological footprint and foster development in socioeconomically disadvantaged regions.
The initial conception and its subsequent design aimed to provide a solution to be manufactured by rural or women's textile cooperatives in Brazilian semiarid region, with raw material being provided by garbage collectors cooperatives. Thus, it would be possible to make good use of the local population’s handicraft skills and the textile productive arrangements already installed, in addition to providing extra income for the workers involved.
In terms of technology, PETI-BR is an innovative system composed of discarded PET bottles connected to hose segments and with synthetic fabric strips interconnecting the inner part of the bottle with the subsurface to be irrigated, like a “wick” over the roots of cultivated plants. The basic procedures are detailed in (NASCIMENTO et al., 2016). Regardless of the production scale of the manufacturing process conducted by the artisans, the system can be implemented by anyone with very simple training.
The project was conducted in multidisciplinary and cooperative basis, involving different departments of two universities, one in Brazil and other in Portugal, and two non-profit organizations located in Brazilian northeast region, with large on alternative technology, youth education and familiar agriculture. Although there is not participation in innovation networks nor the involvement of the complete set of possible stakeholders, cooperative practices are intense and reach a high mid-level considering the project nature.
Investigative procedures involved detailed studies on the semiarid biome, needs of the local population, and physical properties of fabrics, specifically regarding capillarity. The inventiveness of the system's construction technique was strongly inspired by reusability principles and adaptability of available resources. The experimentation took place both with the construction of a system prototype and with the application in a pilot batch of cassava crop. These investigative and experimental procedures strengthen the development process and learning.
Several fields of knowledge within the project’s research lines are open to cooperation and the results are shared indiscriminately between researchers, technicians, collaborators and obviously participating institutions. The project may also be replicated by anyone interested in deploying it non-commercially, in social projects.
In short, it is a technological project focused on several dimensions and perspectives: economic, social, political, environmental and humanitarian. Table 3 depicts the metrics obtained for the indicators presented in Table 2. The coordinates for positioning the project in a quadrant map are obtained by applying (1) and adopting the following weighting criteria: p1, p2, p5, p6 = 1/6; and p3, p4, p7, p8 = 1/3.
|
Table 3 - PETI-BR Project: evaluation of R&D quality. |
||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||
|
Source: The authors, 2021. |
||||||||||||||||||||||||||||||||||||
The basic information in the above assessment was found in the project’s publications and documents, and was also complemented by consulting or interviewing the project manager and researchers of the project’s team. By observing the metrics in Table 3, one could conclude that PETI-BR project has clear principles of sustainability, and is able to foster sustainable development on isonomic bases. Currently the project is evolving to a new version, with improvements in its irrigation mechanism, in order to make it more efficient and to facilitate the system's operating handling.
Case 2: CERTICS
CERTICS is a software assessment methodology. More precisely, it is an innovative model and method of assessment based on the identification of evidence and indications of the creation and expansion of competences in the place of software development. This qualitative approach allows the evaluation to be adapted to the reality of each company, while maintaining the traceability of acquired competence. An overview of the design, main characteristics and first results of CERTICS are shown in (ALVES et al., 2014).
Such assessment methodology was designed and developed in multidisciplinary basis, involving researchers, specialists in various disciplinary fields, technicians, and project managers. Exploratory interviews with stakeholders and expert panels were adopted as investigative procedures, and those interviews also included the project team and academy representatives. The knowledge thus obtained, added to the analysis of scenarios and research on innovation economics, was gradually systematized in a certification model.
The innovation of the methodology was strongly inspired by principles of adaptability regarding different sizes and types of software companies, and considering the political scenario and social development by means of acquisition of skills in the country. The experimentation and project validation was carried out with the construction of a model that was tested in a pilot application with 15 software companies, and the results of such application generated new versions of the model.
The project’s results were presented and discussed in successive meetings with stakeholder groups. From such interactive practice, further elements were collected to improve the methodology. Those results, in turn, gave rise to a new version of the model and method of assessment that came to compose CERTICS methodology. As last stage of development, the methodology was submitted to a public consultation, with wide participation of parties interested in its deployment.
CERTICS is a technological project focused on several dimensions and perspectives, and developed by means of cooperative practices. It is currently in operation, responding to demand for certifications of software developed in Brazil. Table 4 depicts the metrics obtained for the indicators presented in Table 2. Similarly to the study case 1, the coordinates for positioning the project in a quadrant map are obtained by applying (1) and adopting the following weighting criteria: p1, p2, p5, p6 = 1/6; and p3, p4, p7, p8 = 1/3.
|
Table 4 - CERTICS Project: evaluation of R&D quality. |
||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||
|
Source: The authors, 2021. |
||||||||||||||||||||||||||||||||||||
The information that enabled this evaluation was found in the project’s publications and documents, and was also complemented by consultations or interviews with researchers who were part of the project team. Based on the metrics in Table 4, it is possible to conclude that CERTICS has clear principles of sustainability. Since it came into operation, CERTICS project has already enabled the realization of 44 certifications of software produced by 37 companies based in Brazil.
Visualizing evaluations on quadrants map
Figure 4 depicts the positioning of the two evaluated projects in a quadrants map. Note that both projects are in the sustainability quadrant, with a slightly more favorable position to PETI-BR. Nevertheless, it is important to remember that the visualization by means of the map has no effect of providing a simple and direct comparison between projects, but rather of evaluating the current conditions of the project and its evolution over time.
|
Figure 4 - Quadrants map of the extended-ecological view evaluation. |
|
|
|
Source: The authors, 2021. |
7 Conclusions and final remarks
This type of approach can be understood as a supplementary analytical effort to evaluate the impact of cooperative processes and methodological multidimensionality adopted in R&D projects. This analytical deepening applies to projects that have already reached a certain condition of maturity in terms of innovation and technological capacity building.
Such an approach brings the analytical focus to the level of sustainability that a project might achieve or has achieved after it being created and implemented. In this sense, the focus on cooperation and sustainability raises the analysis beyond the aspects of market competitiveness that traditional approaches prioritize. Innovation and the competitive differential are now modulated by an extended concept of ecology, in which the relations among communities and environment, balanced exchanges and evolutionary equilibrium between all parts of the ecosystem become natural goals, as a living system.
The two case studies make visible the analytical potential of this approach, both in terms of supplementing other specific methodologies for assessing projects and impacts of RD&I, and in accommodating under the same analytical look, albeit comprehensive, projects with significantly distinct nature and applicability. To some degree, the broad spectrum of applications in which such approach may be adopted illustrates the pertinence of using the term “extended ecological perspective” as a helpful metaphor for supporting the development and communication of the proposed methodology.
As a new perspective to the integral contributions of R&D efforts, an expanded vision is thus created in order to observe how projects’ embedded technological innovations are perpetuated. Some methodological concerns were observed with the objective of safeguarding evaluation results, mainly by commending the epistemological principles of complexity theory and to place the approach in the always-delicate confluence of methods of the exact and human sciences.
However, even with promising initial results, this approach is still being developed and requires additional studies to overcome some of its limitations. Projects’ data availability is a constraint in developing certain metrics; therefore it should be contemplated in future designs and policies, in order to allow the project be completely evaluated by this methodology. Such constraint refers more to new indicators that could be included in the basic group presented here and to an application of the methodology to an extensive number of projects, with statistical analysis.
Sectoral and statistical-based studies are also envisaged for this type of approach, but we believe that its greatest analytical potential lies in using it for semi-qualitative studies to support decision-making processes and to improve projects’ quality. And more, one may adopted this approach as an element of a “sociotechnical-therapy" script for ex ante analyzes with the project team, in order to detect deviations from the initial objectives and "pathologies" typical of a project in its initial phase of development. In this sense, analyzes produced by means of this approach can provide inputs to other methods of evaluating technological innovations in ongoing R&D projects or retrospectively, such as the socio-techno therapy used by PROTEE or SOCROBUST methodologies (JOLIVET et al., 2002). In case of retrospective analyses, the main contribution is to broaden the knowledge base and guide future projects or even R&D policies.
When cooperating with other analytical methodologies, the extended-ecological approach can play a complementary or supplementary role. The dialogue between methodologies, analysts and project members contributes to the learning process. By applying this approach in isolation or in cooperation with others methodologies, it is very likely that information necessary for evaluating some indicators will not be found in project documentation, and will have to be raised together with the project coordinators through direct consultations or participating in work meetings. Such dialectical praxis favors the learning process, in terms of project planning and execution, thus strengthening the construction of historical series to guide new projects and support the formulation of RD&I policies.
With regard to the feedback resulting from these assessments and the direction of new projects so that they can make progress toward higher levels of sustainability, special attention is required in some circumstances. By increasing the number of dimensions and perspectives, as well as cooperative practices, one must ensure that the project does not become complicated, thus hindering its execution – remembering that complexity and complication are not synonyms, but the former can lead to the latter if not properly treated. Although the complexity expressed by multidimensionality can contribute to innovation and sustainability, its sudden increase may exceed the project's manageability, introducing obstacles that, in large numbers, could make it inexecutable.
Similarly, the involvement of many stakeholders and the participation on many cooperating efforts, such as innovation networks of varying scope, can make learning processes less efficient due to difficulties in knowledge management. In this respect a balanced growth of complexity is recommended, in line with the available R&D resources and the ecosystem’s capacity to absorb the resulting innovation.
The present methodology can also be used as a dynamic and moving map, in which regular assessments can reveal the evolution of the project over time, providing a record of the learning curve and an overview of the consolidation of sustainability principles in the project’s configuration. In addition, the concepts and principles that serve as lenses to the extended-ecological point of view apply to this very approach, recursively and recurrently in an evolutionary spiral, allowing the approach to learn from the results of its own application and thus self-innovating. On such aspect, it is possible to weave one more analogy with living systems, the autopoiesis, as was suggested earlier in similar context.
In this sense, another possible future work is the evolution of this approach to refine the assessment of projects that reached the fullness of dimensionality and cooperation aspects. Such analytical deepening may include, for example, indicators to assess the extent to which participatory management is practiced, and a breakdown of technologies and methods of cooperation that can strength the construction of knowledge. Furthermore, systematic exercises with different weighting factors should be carried out in order to evaluate the related effects and the method sensibility. Depending on such results, the criteria for choosing the parameters may be restricted or be universal, so that a specific weighting context does not overlap with a general evaluation criterion, thus safeguarding the accuracy of the evaluation methodology.
Innovation and learning, while behaving as agents of technological change, interrelate dynamically in an endogenous process of cooperation and growth. All this movement can be ultimately understood as a continuous process of learning, from which it is possible to generate new knowledge and leverage sustainable innovations. As in any historical process, the built-in knowledge base is essential to projects’ development, correcting trajectories and cementing the methodological and creative processes. Incorporating new perspectives and, when necessary, exploring a road not taken, contribute to both learning and technological innovation, axes of this analytical approach and pinnacles of organizational and societal change.
Based on what we could apply and analyze, as well as foreseeing the next steps of methodological development, we conclude that this approach has analytical potential to identify specific aspects of innovation processes and can therefore contribute to making R&D projects more effective. For this reason, it is opportune to elect a holistic and supradisciplinary point of view to evaluate the intensity and results regarding learning and technological innovation. The comprehensive concept of cooperation, the notion of ecosystems as a link between technological culture and nature, and the design of projects according to multidimensional perspectives create the foundation for innovation and sustainable development, in a continuous and more “solidary” way.
Acknowledgements
The authors are grateful to the Brazilian National Council for Scientific and Technological Development (CNPq) for the support, in the form of research scholarships, which allowed this work to be carried out. They also thank the team of evaluated projects and the researchers who have contributed valuable suggestions.
REFERENCES
ALVES, A. M. CERTICS Assessment Methodology for Software Technological Development and Innovation. In: INTERNATIONAL IEEE QUATIC CONFERENCE, 9, 2014, Guimaraes, Portugal. 2014. Conference Preoceeding [...], p. 174-177.
ALVES, A. M. Proposta de uma estrutura de medição para qualidade do SPB: Software Público Brasileiro. Tese de doutorado – Escola Politécnica, Universidade de São Paulo, São Paulo, 2013. Disponível em: http://dx.doi.org/10.11606/T.3.2013.tde-11072014-001021. Acesso em:
ANTONELLI, C. (ed.) Handbook on the economic complexity of technological change. Cheltenham: Edward Elagar Publishing, 2011.
BEERS, C.; ZAND, F. R&D cooperation, partner diversity, and innovation performance: An empirical analysis. The Journal of Production and Innovation Management, v. 31, n. 2, p. 292-312, 2014. http://dx.doi.org/10.1111/jpim.12096
BELDERBOS, R.; CARREE, M.; LOKSHIN, B. Cooperative R&D and firm performance. Research Policy, v. 33, p.1477–1492, 2004.
BELLEN, H. M. Indicadores de sustentabilidade: Uma análise comparativa. Rio de Janeiro: Editora FGV, 2006.
BUENO, B.; BALESTRIN, A. Inovação colaborativa: Uma abordagem aberta no desenvolvimento de novos produtos. Revista Adm. de Empresas - RAE, v. 52, n. 5, p. 517-530, 2012. http://dx.doi.org/10.1590/S0034-75902012000500004
CAMARGO, L. S.; BECKER, M. L. R. O percurso do conceito de cooperação na epistemologia genética. Educação & Realidade, v. 37, n. 2, p. 527-549, 2012.
CHESBROUGH, H. Open innovation: Where we've been and where we're going. Research-Technology Management, v. 55, n. 4, 2012. http://dx.doi.org/10.5437/08956308X5504085
CHRISTENSEN, C. The inventor’s dilema. Londres: Routledge, 2003.
COAD, N.; PRITCHARD, P. Leading sustainable innovation. Routledge. 2013
COCCO, G. Indicadores de inovação e capitalismo cognitivo. In: CENTRO DE GESTÃO E ESTUDOS ESTRATÉGICOS. Bases conceituais em pesquisa, desenvolvimento e inovação: Implicações para políticas no Brasil. Brasília: Centro de Gestão e Estudos Estratégicos, 2010. p. 33-68.
EDWARDS-SCHACHTER, M.; CASTRO-MARTÍNEZ, E.; FERNÁNDEZ-DE-LUCIO, I. International co-operation between firms on innovation and R&D: Empirical evidence from Argentina and Spain. Journal of Technology Management & Innovation, v. 6, n. 3, 2011.
EUROPEAN COMMISSION. Science, research and innovation performance of the EU: A contribution to the open innovation, open science and open to the world agenda. Luxembourg: European Union, 2016. http://dx.doi.org//10.2777/427046. Acesso em: 12 jan. 2021.
FAEMS, D.; VAN LOOY, B.; DEBACKERE, K. Interorganizational collaboration and innovation: Toward a portfolio approach. Journal of Product Innovation Management, v. 22, p. 238–250, 2005.
FISCHER, M. M.; DIEZ, J. R.; SNICKARS, F. Metropolitan innovation systems: Theory and evidence from three metropolitan regions in Europe. Berlin: Springer-Verlag, 2001.
FRANCO, M. J.; BARBEIRA, M. Uma análise conceptual de relacionamentos inter-empresas a partir do enfoque da análise estrutural das redes sociais [A conceptual analysis of inter-company relationships based on the structural analysis of social networks]. In: FRANCO, M. J. B. et al. (org.) Cooperação entre empresas, clusters, redes de negócios e inovação tecnológica. Anápolis: Ed. da Universidade Estadual de Goiás, 2009. p. 39-54.
FREEMAN, C. The ‘National System of Innovation’ in historical perspective. Cambridge Journal of Economics, v. 19, p. 5-24, 1995.
HARLEY, J.; BLISMAS, N. An anatomy of collaboration within the online environment”. In: ANANDARAJAN, M.; ANANDARAJAN, A. (ed.). e-Research collaboration: theory, techniques and challenges. Berlin: Springer-Verlag, 2010.
HOLANDA, G. M.; MATTOS, C. V.; ALVES, A. M. A conceptual and methodological framework for evaluating R&D projects. P2P & Inovação, v. 4, n. 1, p. 141-163, 2017. http://dx.doi.org//10.21721/p2p.2017v4n1.p141-163. Acesso em: 12 jan. 2021.
JOLIVET, E.; LAREDO, P.; SHOVE, E. Managing breakthrough innovations: The SOCROBUST methodology. In: ANNUAL CONFERENCE R&D MANAGEMENT, 2002, Leuven.
KATES, R.W.; PARRIS, T. M.; LEISEROWITZ, A. A. What is sustainable development? Goals, indicators, values, and practice. Environment, v. 47, n. 3, p. 9–21, 2005.
KUPFER, D.; AVELLAR, A. P. Innovation and cooperation: Evidences from the Brazilian innovation survey. In: ENCONTRO NACIONAL DE ECONOMIA DA ANPEC, 37, Dec. 2009, Foz do Iguaçu, Brasil.
LAI, E. R. Collaboration: A literature review. Research Report, Pearson, 2011.
LATOUR, B. Nous n'avons jamais été modernes: Essai d'anthropologie symétrique. Paris: La Découverte, 1991.
LUO, X.; RINDFLEISCH, A.; TSE, D. K. Working with rivals: The impact of competitors alliances on financial performance. Journal of Marketing Research, v. 44, n. 1, p. 78-83, 2007.
MALAINA, A. Tecnología y complejidad: Una aproximación multidimensional desde el “pensamiento complejo” de Edgar Morin [Technology and complexity: A multidimensional approach from the "complex thinking" of Edgar Morin]. In: AGNADO, J. M.; SCOTT, B.; BUCHINGER, E. (coord.). Technology and social complexity Murcia. Universidad de Murcia, 2009. p. 63-79.
MATEI, A.; ANTOINE, C. Complexity theory and the development of the social innovation. Procedia - Social and Behavioral Sciences, v. 185, p. 61-66, 2015. Disponível em: https://www.sciencedirect.com/science/article/pii/S1877042815021643?via%3Dihub. Acesso em: 12 jan. 2021.
MATURANA, H.; VARELA, F. De máquinas y seres vivos: Una teoría sobre la organización biológica. Santiago, Chile: Editorial Universitária, 1973.
MIRANDA, E. C.; FIGUEIREDO, P. N. Dinâmica da acumulação de capacidades inovadoras: evidências de empresas de software no Rio de Janeiro e em São Paulo [Dynamics of accumulation of capability for innovation: Evidence from software firms in Rio de Janeiro and São Paulo]. ERA - Revista Adm. de Empresas, v. 50, n. 1, p. 75-93, 2010.
MOORAR, R.; GALLOUJ, H.; HAMMADOU, H. Public-private innovation networks and innovation activities in French service firms. Journal of Innovation Economics & Management, v. 2, n. 10, p. 191-217, 2012.
NASCIMENTO, N. R.; SALVADO, L. R.; BORGES, F. F.; HOLANDA, G. M. Fabric Waste Applied to Low Cost Subsurface Irrigation. In: FIBER SOCIETY SPRING 2016 CONFERENCE, 2016, Mulhouse - France. p. 177-178.
OECD Fostering innovation for green growth. OECD Green Growth Studies. Paris: OECD Publishing, 2011b.
OECD. Frascati Manual: Proposed standard practice for surveys on research and experimental development. Paris: OECD Publications Service, 2002.
OECD. OECD Innovation Strategy 2015: An agenda for policy action. MEETING OF THE OECD COUNCIL AT MINISTERIAL LEVEL, 2015, Paris. Available from: https://www.oecd.org/sti/OECD-Innovation-Strategy-2015-CMIN2015-7.pdf. Access in: 12 jan. 2021.
OECD. Opportunities, challenges and good practices in international research cooperation between developed and developing countries. Global Science Forum, Report, 2011a. Available from: http://www.oecd.org/sti/inno/47737209.pdf. Access in: 12 jan. 2021.
OECD. Oslo Manual: Guidelines for collecting and interpreting innovation data. 3rd Ed. Paris: OECD Publications, 2005.
PASTOORS, S. et al. Towards sustainable innovation: A five step approach to sustainable change. Marburgo, Alemanha: Tectum Verlag, 2017.
PIAGET, J. Estudos sociológicos [Sociological studies]. Rio de Janeiro: Forense, 1973.
SIERRA, C. Managing global alliances key steps for successful collaboration. Boston, Massachusetts, EUA: Addison-Wesley Pub. Co. 1995.
STACEY, R. Complexity and creativity in organizations. Oakland, CA: Berrett-Koehler Publishers, 1996.
STERMAN, J. D. Business dynamics: Systems thinking and modeling for a complex world. USA: Irwin McGraw-Hill, 2000.
STODDART, H. A Pocket guide to sustainable development governance. Herne Bay UK: Stakeholder Forum, 2011.
STOKES, D. E. Pasteur’s quadrant: Basic science and technological innovation. Washington: The Brookings Institution Presse, 1997.
TETHER, B. S. Who co-operates for innovation, and why: An empirical analysis. Research Policy, v. 31, p. 947-967, 2002.
TORRES, R.; ALMEIDA, S.; TATSCH, A. L. Cooperação e aprendizado em arranjos produtivos locais: Aspectos conceituais e indicadores da Redesist [Cooperation and learning in local productive arrangements]. In: SEMINÁRIO MERCOSUL, 10, Rio de Janeiro, 2004. Disponível em: http://www.redesist.ie.ufrj.br/textos-e-slides-s10. Acesso em: 12 jan. 2021.
TSAI, W. Social structure of "coopetition" within a multiunit organization: coordination, competition, and intraorganizational knowledge sharing. Organization Science, v. 13, n. 2, p. 179-1902002.
UNITED NATIONS. Global Sustainable Development Report. 2015 Edition – Advanced unedited version. 2015. Retrieved from: https://sustainabledevelopment.un.org/globalsdreport/2015. Access in: 12 jan. 2021.
UNITED NATIONS. Report of the World Commission on Environment and Development: Our Common Future. Oxford: Oxford University Press, 1987. Retrieved from: https://sustainabledevelopment.un.org/
milestones/wced. Acesso em: 12 jan. 2021
WALKER, P. Working collaboratively: A practical guide to achieving more. Abingdon UK: Routledge, 2013.
WU, J. Cooperation with competitors and product innovation: Moderating effects of technological capability and alliances with universities. Industrial Marketing Management, v. 43, n. 2, p. 199–209, 2014. http://dx.doi.org/10.1016/j.indmarman.2013.11.002. Access in: 12 jan. 2021