Abstract
Agile software development has large success rate due to its benefits and promising nature but natively where the size of the project is small. Requirement engineering (RE) is crucial as in each software development life cycle, “Requirements” play a vital role. Though agile provides values to customer’s business needs, changing requirement, and interaction, we also have to face impediments in agile, many of which are related to requirement challenges. This article aims to find out the challenges being faced during requirement engineering of agile projects. Many research studies have been conducted on requirement challenges which are somehow biased, no suggestions are given to improve the agile development process, and the research does not highlight large-scale agile development challenges. Hence, this article covers all the challenges discussed above and presents a comprehensive overview of agile models from requirement engineering perspective. The findings and results can be very helpful for software industry to improve development process as well as for researchers who want to work further in this direction.
1. Introduction
Agile software development has many advantages over traditional and linear development models. Up-front and preliminary preparation is sufficient to get all factors for consideration that might influence the development process [1], but think for while is it possible to get all those factors or variables that affect the system? Empirically, the answer should be “no.” To cope change, uncertainty, and unknown factors, an adaptive approach should be used.
Agile is an adaptive approach, and factors like high success rate and remarkable results have made it very popular in software development community. Agile software development (ASD) works best with respect to traditional development approaches and practices due to many reason as rapidly changing needs of customers, urgent business requirements, and flexible nature besides many others. The choose of right SDLC (software development life cycle) while developing software solution is very important task either water fall, V model, or agile all of them have their own advantages and disadvantages [2]. The choice of SDLC approach mainly depends on the nature of the product being developed.
In software development, ”Requirement engineering” (RE) plays a vital role as the whole success of software depends on the correctness of requirements gathered requirement in engineering phase [3, 4]. So, when the nature of requirement is dynamic, it is not a simple task to gather, analyze, understand, and manage requirements [5]. The dynamic nature of requirements and their consideration according to project completion is necessary as we move from first phase of the project to the final phase thus change consideration is important factor and applied in large ratio in project postmaintenance [6]. In traditional sequential methods, the RE phase is freezed and then later there is no user involvement and there is no way to cope with a change. Though agile software development (ASD) is better than other traditional software methods in many ways, i.e., it supports direct team involvement, early release of software, embracing change, cooperate the need, and change of emerging technology, and many other. Although there are so many benefits of agile development, but still proper RE is critical for successful projects. Contrary to the traditional SE process, the agile method relies on iterative development and face-to-face communication.
In software industry, the success rate of project is depended on the correctness of requirements irrespective of how they are gathered? Whether they are gathered correctly or not according to user perspective? According to [7], in agile development, the process of requirement engineering (RE) is not just restricted to the early phase as in beginning such as in traditional models but continues till whole development cycle.
Balaji and Murugaiyan [2] presented a comparative study on SDLC and discussed the pros and cons of waterfall, V model, and agile model. Their findings are if project is small and requirements change continuously chooses agile approach and if project is large and requirements are clear, then waterfall approach is the right choice, and if project is large and requirement changed continuously, then the V model serves the purpose. Due to proven success rate of agile methodology, it is now-a-days being adopted by larger organization for complex and large projects. De Lucia and Qusef [8] discussed how to obtain and define user specification with some results and solution confronted with agile approaches. The Standish Group International CHAOS Survey identified many causes, challenges, and failure factors of software projects. Studies show that the project development challenges are 37% from the requirement phase as illustrated in Figure 1 [8, 9]. The problems and solutions of requirement traceability in ASD and some guidelines for ARE (agile requirement engineering) have been addressed. According to [10], the majority of the software projects fail due to the poor requirements management and due to common challenges which cause the failure of project.
According to Kettunen and Laanti [11], software development companies use agile approaches where small team works very closely with customers to build high-quality software with frequent iterations and feedback. If the nature of the project is small, agile methodology works very well and produces effective products. To gain benefits of agile in case of complex software products and organization, the basic assumptions will not work and there is a need to satisfy additional constraints. Nerur et al. [12] reported the migration challenges to agile. There are many scenarios where companies that are following full or partly agile practices for large-scale or distributed projects had positive impact of using agile in large-scale software projects [13–15]. For example Yahoo, Google, Microsoft, Siemens, CNBC, AOL, and others are using agile in their development projects [16]. There are many principals and guidelines available for using agile methodology for complex and large-scale software development such as the utility of rapid application for the development of large and complex projects [17–19]. Numerous research studies have shown that agile approach has proven effective for large-scale projects [20]. Bano et al. [21] reported the causes of requirement change and classified them into two categories as accidental and essential. For better understanding, McGee and Greer [22] used a taxonomy, and Saher et al. [23] categorized the causes of requirement change factors into reason and change of origin. Agile requirement in large scale is presented in [24] and analysis and management of conflicts between team are described in [25]. The authors in [26] performed a brief comparative study on agile software development approaches. Knaster and Leffingwell [27] applied scaled agile methodology into system engineering and lean softwares. Putta et al. [28] discussed the challenges and advantages of applying agile methodology SAFe (Scaled Agile Framework) and [29] discussed the scaled agile methodology.
Although numerous studies have been done on agile development and adoption of agile practices, there is a limited research on challenges being faced in RE using agile development strategy and their solutions. Mainly those reviews somewhere and somehow are selective and biased in nature. In [19], the authors discussed RE challenges with only limited reviews, and no case study, survey, or industrial experience has been shared. In their scheme of study, there is biasness, and imprecision and the underlying cause of the challenges reported have not been discussed in detail [30]. In [31], the authors have summarized the results and decisions thus leading to biasness. The existing literature lacks the discussion on RE issues for large-scale projects, and limited reviews are available for RE challenges. In addition to this, no improvement and suggestions are described [32].
The limitations stated above motivated us to conduct a literature review on “Challenges of Requirement Engineering in Agile Software Development.” Section 2 presents discussion on agile software developments and requirement engineering. In Section 3, we discuss Requirement engineering challenges for small- to mid-scale agile projects. Section 4 presents requirement engineering challenges for large-scale agile projects. Section 5 focusses on reported challenges, their cause, and potential solutions. In Section 6, generic guidelines to improvise the agile requirement engineering process are presented. Section 7 highlights the limitation of agile development. Lastly, Section 8 concludes the article and provides directions for future research.
According to the literature, many industrial project teams work in an agile environment in order to have rapid delivery of high-quality software for both small- and large-scale projects. The scheme of proposed methodology is represented in Figure 2.
This article presents a detailed review and presents appropriate findings and suggestions. Research aims to find out the problems and challenges being faced in requirement engineering of small-scale, mid-scale, and large agile projects design some possible suggestions to improve the general RE process in agile based on the literature analyze and evaluate the literature in order to propose a suggestion on the future work
The article aims to address the above stated problems with recent published articles in few years using the agile development with main focus on requirement engineering process and to find challenges involved in agile engineering activities/practices.
2. Agile Software Development and Requirement Engineering
Agile itself is not a method but an approach based on manifesto that mainly focusses on 12 principles and 4 core values. The values of agile manifesto are working software with comprehensive documentation, individuals and interaction among tools and process, customer collaboration, and quick response to a change [33–35]. Agerfalk and Fitzgerald [36] stated that agile development techniques significantly differ from the traditional planned approaches by focusing on productivity rather than following the rigourous processes. According to [37], more than 90% practitioners of agile methodology use user stories to capture requirements and [38, 39] provide how to improve the quality of agile requirements.
ASD is an adaptive process with less documentation, limited resources, and small- to medium-sized highly skilled and motivated team members. The application of agile methods enables the development of reliable and usable software using multiple iterations. Agile addresses the issues of the RE traditional development model as the major challenges are related to requirement and due to the change of business process [40]. Agile development overcomes the issues of the sequential model but still there are some drawbacks associated with it [40].
2.1. Agile Frameworks
De Lucia and Qusef [8] and Malik et al. [40] discussed agile models such as Agile modeling (AM), feature-driven development (FDD), extreme programming (XP), and SCRUM from requirement engineering point of view. The overview of these agile models is listed as follows: AM is a technique that provides some guidelines related to design models and is used for keeping the documentation and models as minimum as possible [8]. Requirement engineering approach, i.e., brainstorming is supported by some of AM applications. FDD is a 5-step procedure with some specific entry and exit criteria mainly focused on construction and design phases, building feature list, and planning which are iteratively designed and built by features steps, where feature represents a client valued function [8]. According to Malik et al. [40], the product is developed in terms of individual or separate features instead of whole or full product. DSDM is the extension of rapid application development and has five principles including user participation, consistent implementation, team decision making, comprehensive life-cycle monitoring, and reversible developmental changes. The initial phases implement feasibility and business study to capture baseline requirements, and further requirements are extracted in subsequent development phase [8]. In the development phase, DSDM supports any requirement engineering approach. XP is a famous agile method used for successful development of software and supports ease, communication, feedback, and simplicity despite continuously changing requirements. Some XP common practices are as follows: pair programming, fast iterations with small releases, fast feedback, close customer interaction and participation, and so on. In [8], the authors describe the RE activities in detail. XP focusses on coding and development instead of gathering requirements [40]. In XP, the team passes the accountability for the usability of the product to stakeholders [41]. SCRUM is a flexible, adaptable, and project management approach with 30 day sprint cycle. For coordination and integration purpose, 15 minutes daily meeting is conducted [8]. In SCRUM, the developed product is always in working condition and only functionality of the product is increased after every iteration [40, 42].
According to De Lucia and Qusef [8], RE activities such as discovering, analyzing, specifying, and documenting are the activities of great importance. The RE activities feasibility study, elicitation and analysis, documentation, validation, and management have also been discussed. Kumar et al. [43] presented a hybrid approach for RE in agile system development by using JAD. Requirement prioritization in agile development is done with the help of viewpoints. High-quality projects are developing in less time using agile development methodologies including iterative development, increments of project, adaptation, collaboration, and emergence. For software requirement gathering, JAD session provides customer specification with collaboration of customers and view point technique is helpful in prioritizing software requirements.
2.2. Agile Adoption at Scale
A large development project can be specified with respect to some criteria used by a company or with absolute measures such as number of developers involved, budget size, complexity, or number of development teams [44]. Positive results and success rate of agile implementation in small projects have created the interest of software development teams and industry to adopt agile in large projects [44, 45] as agile adoption is rapidly replacing the traditional methods [46–49]. In [50], the authors describe how environment helps and supports in distributed agile framework.
The adoption of large-scale implementation of software project may cause project management challenges [51] despite of issues the adoption rate of agile at large scale is increasing rapidly [52, 53]. The survey on management of software development projects is conducted in Oslo, Norway, March 2015. The information about 108 software projects has been examined, and their result demonstrates that agile methodology implementation rate over small, medium, and large projects is successful [54]. The results of agile implementation rate over projects are shown in Figure 3.
On the other side, the adoption of agile at scale has some limitations in some cases, i.e., SCRUM is suitable for small-scale projects with less number of team members, and when it is applied on scaled projects, it makes development less efficient [55]. Abrar et al. [55] identified 21 motivators for the adoption of agile at large scale. Furthermore, their analysis and classification such as into critical factors have been done based on defined criteria. Identified motivators are validated by surveys, questionnaires, and with the empirical study in presence and help of agile software development practitioner and experts [55]. Khalid et al. [56] identified the challenges for SCRUM methodology for distributed environment.
Laanti [57] presented 5 stages of agile transformation for large-scale software development. Salesforce.com completed an agile large project with 200 project members just in the time of 3 months [58]. Bick et al. [59] described five ways of practicing agile in large projects. Turetken et al. [60] discussed the levels of the scaled agile implementation and maturity model. The success rate of agile models in terms of acceptability % and success rate% with respect to budget size of the software project is always greater than the rate of partially agile and not agile methods in each small, medium, and large budgeted projects [54]. The facts summarized above prove and make a way to adopt agile at large scale in software projects.
2.3. Requirement Engineering Practices in Agile Software Development
According to Kovitz [61], RE in agile development is carried throughout the entire development cycle but in small sessions of conservations as compared with phased requirement engineering. In [62, 63], the authors focus on description of important key practices for ASD. Characteristics and principles for large-scale agile development are presented in [64, 65]. Some common agile RE practices are described as follows: To benefit from agile process, the stakeholders and the team members should communicate directly [66]. For successful project and to avoid failure, the main contributing factor is the need for consistent and prioritized requirements by the customer. This emphasizes on the need of client participation and interaction [67]. Requirement emergence, prioritization, and change management including updation of features is important [68]. Continuous planning is required to cope with new and changing requirements, and requirement engineering activities are performed with shared concepts [69]. Use prototype to get feedback from client for requirement refinement and prioritization [8]. Remove an outdated requirement, and for each iteration, prioritize the requirements from beginning [5].
Test first, refactoring, pair programming, small releases, and on-site customer are the fundamental practices that are important to understand the complex skills [61]. Other than this, breaking big things into tiny things, giving up control, writing meaningful test, communication, object oriented design, and usage of advance tools for fast development cycle are considered as important skills in agile development [61]. The common challenges faced in agile projects during the requirement engineering process along with their activities are identified and presented in Table 1.
Rodriguez et al. [70] described some findings related to requirement engineering activities of agile system development. The reported findings are limitations in success of agile development. The intention of this project is to develop a product, so for development of product, some previous details are available. Nonfunctional requirements and management of requirement interrelationship are major issues. A case study of TOPEN (Test Operation Environment) primer development including features, scope, and process by using conventional methodology is briefly discussed. This architecture has four components as TOPEN Engine, Mission Information Base (MIB), gateway, and a GUI TOE. Testing and monitoring of the complex system are done in a specific domain environment by using the conventional method. A list of existing and dropped features of initial product as well as a list of some new functionalities is presented. There are many issues in conventional processes, so they can be more critical for agile development processes.
3. Requirement Engineering Challenges for Small- to Mid-Scale Agile Projects
Agile development overcomes the issues of the sequential model but still there are drawbacks associated with it [40]. According to Malik et al. [40], the challenges of RE are incompatible interfacing, negligence of nonfunctional requirement, lack of unambiguous, and RE activates. Agile approaches do not have any proper framework by which the user requirements are properly listed or documented, and due to changing requirements, extensive amount of work needs to be done repeatedly. Sometimes due to changing needs of the customer, the developed product in previous iteration causes interface compatibility issues. There is no rule or technique for gathering and evaluation of nonfunctional requirements. The quality of the product being checked by the user’s feed back after iteration is complete [40]. Sebega and Mnkandla [6] discussed the issues of South Africa software industry. The challenges reported by the authors are as follows: In requirement elicitation: lack of clarity, requirement prioritization, and problem scoping In requirement management: prioritization issue In requirement documentation: lack of proper documentation and no customer representative is available In requirement validation: no techniques and tools for validation process In requirement management: lack of requirement change management and appropriate management tools
Lack of proper documentation may create some issues for the team; hence, De Lucia and Qusef [8] suggested some guidelines for their long-term use. These guidelines are assignment of some people to produce little documentation, use of computer tools for modeling, and development of reverse engineering techniques. According to Alam et al. [19], less direct communication and insufficient documentation cause requirement tracking issue and less feedback and interaction of client, handling requirement change causes more work and estimation of time and goes out of plan, and creative and innovative requirement in agile itself is challenging, missing, and conflicting requirements which causes large number of iterations and no focus on nonfunctional requirements. Hajjdiab and Al Shaima Taleb [16] identified the challenges faced in SCRUM. The challenges faced in requirements are no team involvement in initial project estimation, impact of ambiguous requirements on quality and schedule, minimum documentation, different levels of abstraction, and inconsistency of user stories in product backlog when product owner is appointed. According to Amir et al. [71], due to changing requirement, the agile project becomes expensive, so project cost and maintenance cost are increased. Minimal documentation in requirement gathering process creates issues such as no staff help and provides no support for reverse engineering practices. Lack of documentation creates other issues when the project is transferred to maintenance, when staff changes, or when new team or staff is hired [72].
Cao and Ramesh [68] stated that in agile approach, the RE focusses more on requirement validation but there is no formal verification of requirements as there is no model for formal verification of detailed requirements in agile. For requirement prioritization, the only criteria are business value as time to market is important factor which can create major issues later. In agile RE, the use of prototyping creates the wrong understanding in the stakeholder’s mind about development speed. Too much implementation in initial prototyping causes issues such as quality concerns generated by the practice of reusing code in prototype, and stakeholders may not accept other development cycles which are more scalable and robust. According to Ganesh and Thangasamy [73], the challenges of RE lie through clear communication and information sharing, domain knowledge, and less documentation on software requirements. Primary reason is that the client and whole team are no longer available for daily face-to-face meetings. No regular communication is possible, so minimal specification causes many issues such as difficulties for quality assurance team and loss of knowledge either domain or product.
Cho [74] discussed the challenges and issues of SCRUM (agile development model) through a case study of a company that used SCRUM in their web-based project from small- to mid-sized projects. In reported issues, the developer said that tricky code requires more explanation and for the changes they made. Whereas many developers said they are easy without documentation. Developers stated the issues regarding the importance of specification, such as no shared knowledge, what if the person who has all information leave company, and some other important team member [74]. Stakeholders are not involved in the decision-making process, and the customer is not readily available for regular meetings which leads to a communication gap. According to Helmy et al. [75], agile approach relies on the basic assumption that getting and eliciting all the requirements from user as they evolve with time so customer changed their requirements and mind. Thus, no one knows full requirements of the product at start of the product that causes missing an interface between requirements and causes more rework in next iteration. Other reported challenges are lack of RE activities and no focus on nonfunctional requirements [75]. According to Helmy et al. [75], the issues related to nonfunctional requirements should be identified and team should adopt the nonfunctional requirements to improve the overall product quality in each iteration; otherwise, it may also cause security and scalability-related issues, which will deeply affect the final version of the product.
According to Daneva et al. [76], in requirement prioritization, users play an important role as delivery story which include important technical details and which are created with collaboration of users. Thus, inability of user such as users are not able to tell exactly what they need and what they actually want to say due to high authority pressure, environmental facts, or hesitation and less representation among their group create issues in creating delivery stories. In agile, customers are not involved in decision making process and it creates concerns related to requirement prioritization. Rahim et al. [77] stated that requirement prioritization in agile approach after each iteration is challenging in absence of any technique, and when each project requirement demands specific consideration of each aspect, it causes starvation problem. Shameem et al. [78] stated that due to rapid development rate and less development cost, software organizations are focusing on applying agile methods in distributed environments which is known as distributed software development (DSD). This study involves distributed agile practices and is focused on identification of intended challenges and their priorities in context of agile development in distributed environment. Literature review was conducted to find out these challenges, and they were further divided into four categories such as team, process, current technology, and management. Comparison of challenges of two studies is performed, and prioritization of challenges on major four categories is done by using the APH method.
4. Requirement Engineering Challenges for Large-Scale Agile Projects
The use of agile strategies is being so popular in software industry due to its numerous benefits. In complex software development projects as supply chain management having unstable and uncertain requirement, development is difficult if the predictable process model is used [79]. As agile strategies are adaptive, they can be used, but using the agile methodologies helps and improves one aspect in large-scale software development (LS-SD) while creating issues in another part. Mishra and Mishra [79] reported RE-related issues adopting agile practices in LS-SD, such as long duration of RE due to complex decision making process, difficulty in creating and maintaining requirement priority list, and waiting time of requirements. Dikert et al. [80] discussed the challenges and success element that influence large-scale agile transformation. The reported 35 challenges are categories in to lack of investments, implementation challenges, RE challenges, quality assurance challenge, and others. The listed RE challenges are missing high-level requirement management, refinement challenges, gap between short- and long-term planning, and hard to create and estimate user stories [80]. In large development projects, the understanding and management of high-level requirements are necessary. So, handling high-level requirements and understanding are important as it is well known that ambiguity in requirements affects the quality. Bjarnason et al. [81] presented a case study of CASE company which develops embedded systems with 60 to 80 new features, 700 to 1000 detailed requirements, and with lead time of 2 years. Agile practices, their advantages, and challenges being in agile requirement engineering (ARE) are reported. The reported challenges using agile practices for large-scale software projects are planning, weak requirement in start, weak requirement prioritization and effort estimation, quality issues, customer proxy role, taking innovation, ensuring RE, validation and verification, motivation for requirement work, lack of requirement documentation and less preliminary planning, and focus that can lead to rework [81]. Missing the scope of requirements in start causes the difficulty in estimation of project schedule and budget.
Large-scale agile projects with four iterations and some specified roles using the SCRUM model that aimed to be more flexible and to avoid specifying details are signed by GOV in order to modernize their existing legacy system [82]. According to Rolland et al. [82], in this replacement project, a major challenge is to scale product owners so that they manage to facilitate the requisite ”translation” of user stories, which are usually multiple, including complex interdependencies and relatively high-level development teams. According to literature [7], in agile development, the process of RE is not just restricted in beginning such as in traditional models but it continues till whole development cycle. Fægri and Moe [7] presented a finding from a case study of large agile-based project. According to Xu [83], in continuous changing requirements, technology, and environment, there is a need to focus on agile methodologies to design and develop a software on time with less defects and early releases thereby achieving customer satisfaction. Xu [83] described coordination challenges faced in large-scale agile projects and discussed the coordination in terms of decision making, communication, and control mode. As the decision making process is difficult for developers using informal way of communication, the problems faced are participant’s lack of interaction, communication problems, information loss, difficult and unstable requirements, complex interdependence tasks, and technical problems.
Kasauli et al. [84] have discussed requirement challenges through multiple case studies of four companies for large-scale system development. Among those, two are car manufacturers, one is telecommunication company, and last one is technology-based company. In LS-SD, scope of the agile method and role of RE are briefly discussed. Different cases of companies are discussed, and result of each case is concluded. Main challenges of requirement engineering of agile system development for large scale of case 1 are related to communication and knowledge management. Other challenges are also discussed which are related to two areas of requirement knowledge such as shared value of user and creating and maintaining system understanding for Case 2. In Case 3, main challenges found are related to interplay of stakeholder from different domains. Those domains are customer, integration, and testing and development domains. In studied case of case companies, requirements specification is needed to better understand the current system for analysis of new feature requests and maintenance of the system.
Lindvall et al. [85] demonstrated that organizations have identical needs and need to experience effect of change before adopting new methodologies in large organizations. This is just because of their complex nature and need to transform current technologies and processes with existing technologies. In [85], the authors tried to establish their results by analyzing their experience in specific framework of large organizations by using well-structured processes. Experience collected on basis of analysis is then shared among SEC members which are recognized as Nokia, ABB, Daimler Chrysler, and Motorola, and their areas of interest are also discussed in detail. Moreover, data across 15 pilot projects of these four organizations are affected by extreme programming that gives a wider overview of implementation of agile methods in large companies. Issues related to implementation of agile in large companies are also discussed. Developers identified requirement problems as powerful drivers among these four organizations. It is difficult to split requirements into detailed specifications on delivery of these requirements to high-level development teams.
5. Reported Challenges, Their Cause, and Potential Solutions
According to Lee et al. [86], the problems with software are as follows: (1) it is rarely delivered on time due to schedule slippiness which causes cancellation; (2) poor softwares which are not able to solve the right problem due to the reason that the software requirement is not according to the need and does not conform; (3) staff burn-out when they leave or when areas of interest; and (4) small changes require huge effort. The following are the challenges we have identified through the literature; and their summary is presented in Table 2; and Table 3 presents a summary on RE challenges faced in large-scale agile development. Figure 4 shows the summary of this section.
5.1. Less Direct Communication with Clients/Stakeholders
Less direct communication causes requirement trackness issue [19]. It may occur due to many factors such as lack of time, distance factor, less customer representatives, and so on. From business perspective, customer involvement and availability directly affect the requirement change and validation. In decision making process, no user involvement causes requirement prioritization. Missing high-quality user interaction results in wrong requirements [68, 96].
How to Deal. To deal this issue, direct communication, surveys, and interviews should be conducted. While gathering requirement from client do not focus on collecting long-term requirements and do not go with long formal interview sessions [40]. Use surveys extensively to get feed back from customers.
5.2. Minimum Focus on Documentation
Lack of proper documentation may create some issues for development team [8] and cause requirement trackness issue [19]. Minimal documentation in requirement gathering process creates issues like no staff help and provides no support for reverse engineering practices [71, 72]. Minimal specification causes many issues such as difficulties for quality assurance team and lack of knowledge either domain or product [73]. This will create more impact on large projects.
How to Deal, Do some documentation, and more focus and consideration should be taken as this will help to track, mange, and test the requirements. Hence, there is need to standardize the agile process and RE activities.
5.3. Missing, Ambiguous, and Conflicting Requirements
The different levels of abstraction and inconsistency of user stories cause problems. So, the impact of ambiguous requirements on quality and schedule is very severe [16]. Missing interface between requirements causes more rework in next iteration [75].
How to Deal. The use of more formal ways for specification of requirements is needed to incorporate the missing, ambiguous, and conflicting requirements. Use more formal ways for specification of requirements to incorporate the changing requirements. RE-KOMBINE is a model that is used to specify requirements in formal way and is more flexible to handle change [87].
5.4. Problem of Prototyping
According to Corral and Fronza [97], we can choose design thinking based on empathize, define, ideate, prototype, and testing than typical approaches. The RE agile development and the use of prototyping create the wrong understanding in the stakeholder’s mind about development speed. Too much implementation in initial prototyping causes issues such as quality issues generated by the practice of reusing code in prototype and stakeholders may not accept other development cycles which are more scalable and robust [68].
How to Deal. Avoid much implementation in initial prototyping. It is better to use paper prototyping instead of wizard of Oz prototyping which actually reduces your effort and remove the misconception of user by seeing wizard prototype that you implemented all so fast. Paper prototyping in this case will help in many ways such as acts as a design test before you code, quickly changeable, and eliminates the specific variables of technology [98].
5.5. Less Preliminary Planning, Focus, and No Initial Team Involvement
In start of development, RE with less preliminary planning and focus can lead to rework; moreover, missing the scope of requirements in start causes the difficulty in project estimation of schedule and budget [81]. No team involvement in initial project estimation causes wrong estimation of project cost and schedule [16].
How to Deal. It is better and helpful to engage and involve all team members of the project from the very beginning such as development team so that they know all about the project specification, their time to time changes, and technical details from the start. The one and important reason to engage and focus on all the teams members is that since there is minimum documentation, their preliminary planning, focus, and involvement are very beneficial. It is the responsibility of the project manager to monitor the presence of team members mentally by engaging them into multiple activities during questionnaires and interviews.
5.6. Less Experienced and Skilled Team
According to Hufmann and Lehner [88], requirement miss-interpretation, miss-communication, and inadequate requirement analysis are the result of less experienced and skilled team which in turn causes low rate of project success and other problems. According to Morales-Ramirez and Alva-Martinez [89], the persons/team involved in activities build wrong feature software products due to less experience and lack of knowledge [90].
How to Deal. Train practitioners to improve their skills by giving them materials, training sessions, and guidelines on how to perform on particular thing [89]. The training plan is applied on practitioners working at a Mexican SDO with grouped set of skills. Moodle platform is used as a supportive tool.
5.7. Inability of Customers in Telling User Stories
Inability of customer while describing user stories causes rework and creates load and extra charges because of incomplete domain knowledge among user groups [76] and hence creates issues in requirement prioritization.
How to Deal. To overcome this problem, we have to train the customer and involve them in decision making process throughout the process of requirement engineering.
5.8. Tacit Knowledge
The knowledge transferred by the business to the development team is a challenge because this information is not listed in specification [73]. Tacit knowledge is what we know but we cannot tell; it is what is known and acquired through personal experience, so sometimes it is difficult to articulate them [99]. According to [100], in tacit knowledge, there is risk associated with requirement misunderstanding and anticipation, and it is very difficult to involve the stake holders from every culture, field, and profession.
How to Deal. Direct communication, observation, surveys, and interviews frequently reduce the likelihood of tacit knowledge. According to [100], the three factors such as domain expert, communication process, and audience will help to encounter the issue of tacit knowledge. It is more appropriate to break the meeting into smaller sessions as it will help participants to digest knowledge between the session and they feel more comfort. The receiver of knowledge (audience in meeting) must have good listening skills [100]. Enable practice for knowledge sharing of positive experience as well as negative experience which helps team members and they will learn from it as sharing knowledge is a key success factor in development [101].
5.9. Changing Requirements
Sometimes due to changing need of the customer, the developed product in previous iteration causes interface compatibility issues [40]. Due to changing requirements, the agile project becomes expensive; thus, the project cost and its maintenance cost increased [71]. Failure towards managing the changing requirements causes system failure [102].
How to Deal. Kamal et al. [91] proposed ARCM-RM (agile requirement change management readiness model) which comprises of three main components such as maturity level, factor level, and assessment level. This model also supports global software development for measuring and improving ARCMRM. Kamal et al. [92] identified 21 key success and progress factor of ARCM, and the findings and results show that they are beneficial for practitioners for management of change in global software development context. Moreover, use automated techniques and tools to manage requirement issues, i.e., JIRA and RE-KOMBINE [40, 71, 87].
5.10. Requirement Prioritization
Requirement prioritization in agile approach after each iteration is challenging due to absence of any technique [77] as stakeholders are not involved in decision making process and customer is not readily available for daily meetings [74]. According to [103], there are almost 4 weeks or more time lapse between the delivery of each deliverable in incremental approaches. In this time period, change in market demand or requirement changing due to personal reason, these factors affect the process of requirement prioritization. There are many unseen and unobserved factors in requirement prioritization, and they have great impact and influence on the result of prioritization [104].
How to Deal. AbdElazim et al. [93] presented a framework to prioritize the requirements in agile development. The framework consists of four stages: identification, verification, estimation, and prioritization. The presented model will also help to handle change in any stage of the software development process [93].
5.11. Negligence of Nonfunctional Requirements
There is no rule or technique for gathering and evaluation of nonfunctional requirements (NFRs), for example, agile methods, extreme programming, and SCRUM are popular as they deliver quality functional requirements. The quality of the product being developed is checked by the user’s feedback after each iteration is completed [40]. Negligence of nonfunctional requirements delays the project delivery as well as deeply affects the quality of the whole product.
How to Deal. Farid and Mitropoulos [67] proposed the java-based simulation tool NORMATIC for nonfunctional requirement modeling in agile development process. Planning and visualization procedure is presented to implement the NFRs in agile projects [94]. Younas et al. [95] discussed about the user story cards, their attributes, and agile NFR elicitation process. The authors [95] analyzed the process of elicitation of NFR by conducting a case study on master students of SED (software engineering department) and obtained the encouraging output.
6. Generic Guidelines to Improve the Agile Requirement Engineering Process
Many guidelines and techniques are presented by many researchers such as agile adoption techniques, agile principle, guidelines, and assumptions for enterprise or large distributed projects and others [82, 105, 106]. Some of them perform analysis and listed success factors [107]. In Figure 5, some generalized suggestions are shown to improve the process of requirement engineering which will help to incorporate challenges of agile development. The generic guidelines are shown in Figure 5.
6.1. Direct Communication, Surveys, and Interviews
While gathering requirements from client, do not focus on collecting long-term requirements and do not go with long formal interviews sessions [40]. Use surveys extensively to get feed back from customers. Frequent face-to-face communication from very beginning for identification of requirement is necessary and keeps the project from cost and schedule overrun [108]. According to [109], when we choose and increase reliance on linear communication medium, accordingly the software defect rate will also increase. So, there is a need to pay more attention while choosing the communication medium when stakeholders are partially available.
6.2. Motivate Developers
Gather and share positive and successful stories and experience to motivate developers [16]. Motivation is considered as an important factor for the developers’ productivity and is very difficult to quantify [110]. Highly motivated developers are more productive, and thus there are more chances of project success and vice versa. Hence, it can be concluded that the project outcome is correlated with motivation [110]; therefore, it is the responsibility of project manager to motivate the developers by giving them credits, appreciation, value, and regards. To motivate developers, Arai et al. [111] proposed a gamified tool to remove warning which helps in detection of bugs in early software development.
6.3. Brainstorming and Prototyping
To cater the issue of creative requirements from customer, individual and collaborative brainstorming techniques are used [40]. According to [96], the use of prototyping can solve many RE challenges as unambiguous documentation by developing executable prototypes, provide motivation of RE as updating prototype is easy rather than documenting, and improve communication among stakeholders as it serve as a visual medium and can capture real needs and results.
6.4. Observe and Analyze Similar Systems
To handle requirement change, observe and analyze the requirements of similar products that are already built because sometimes there is no enough time to brainstorm, gather, and analyze the requirements [40]. In software development industry, projects are related to one another to some extend by their nature or by features; hence, observation and analysis skill will help in many ways. If you have already experienced and developed some similar systems, then you have greater ability to tackle the issues, i.e., reusability of already built components and reusability of requirements and lot of others. Observing and analyzing similar systems will improve the ideas and the skill sets of persons.
6.5. Use Case Scenarios
Agile framework and methodologies use simple way of requirement gathering and specification in the form of user stories, which are easy to read but difficult when we talk about visual representation and the understanding of the system [112]. Wautelet et al. [112] bridged user stories to the use case (UC) model. Although it is very lengthy process, in agile, it can be very effective to use user stories and use case scenarios to capture requirements if customer is readily available [40].
6.6. Decision Making Process
Improve the decision making process and involve stakeholders in decision making process to avoid requirement prioritization issues. Use the requirement prioritization model and techniques in agile for better understanding and for ease of use which are scalable, save time, and customizeable [77].
6.7. Requirement Management
To ensure the integrity of the user information or requirements in agile, there should be proper mechanism or little documentation should be produced as in agile there is no focus on documentation and specifications. The process model for requirement change management should be used as it provides better communication, user understanding, process improvement, and management [113]. Requirement management metrics should be used [10], and tool support for large-scale agile projects should be used [114]. According to Hannay et al. [115], to handle and structure requirements, service-oriented capability taxonomies should be used.
Albuquerque et al. [116] stated that there are a lot of ways to enhance the requirement change management in agile system development. They further proposed a study which describes agile requirement change management (ARCM) process and activities associated to this process in order to manage the requirement change in agile modeling. A systematic mapping is used to control this process and identified 11 different agile practices to proceed the process. Process of ARCM has 3 main steps in which first is identification of change, second is analysis of identified change, and third is cost or effort estimation. These processes are further discussed in detail, and each process has distinct requirement engineering practices which help ARCM to enhance change in requirements.
6.8. Requirement Specification and Change Handling
Use more formal ways for specification of requirements to incorporate the changing requirements. RE-KOMBINE is a model that is used to specify requirements in formal way and is more flexible to handle change [87]. JIRA is also helpful to deal with agile requirements challenges and is more helpful for complex projects [117].
According to Albuquerque et al. [116], the process of agile requirement change management consists of requirement change identification, analysis, cost, and effort estimation to handle and manage change in requirements. AbdElazim et al. [93] presented a framework to prioritize the requirements in agile development which consists of four stages: identification, verification, estimation, and prioritization. The presented model will also help to handle change in any stage of the software development process [93].
6.9. Detection of Quality Defects in Requirements
AQUSA (Automatic Quality User Story Artisan) software tool based on natural language processing (NLP) should be used to find quality defects in requirements as it detects defect in requirements and suggests some possible solutions. Defect detection in early iterations will prevent the defects to appear in next iteration, and therefore we get better quality product [118].
6.10. Standardize the Process and Techniques for Nonfunctional Requirements
There is a need to standardize the agile process and RE activities and to provide a way to elicit and manage the nonfunctional requirements in order to improve the overall product quality in each iteration.
6.11. Use Some Balanced Approach
The reported solution to deal with common challenges faced in large-scale project is to create a balance between the agile and traditional approach. In large and complex distributed projects, agile without structure can cause chaos particularly where planning, control, and teamwork are crucial. Structure without agile concepts can result in rigidity, especially when a project needs extensive learning, innovation, and change [119]. According to [120], documentation in agile development is important to support the communication with stakeholders and it has more importance in distributed development and conversations. Hence, agile methods with zero documentation are not recommended. According to [83], in large agile projects, as the scale of the projects grows, more formal approaches are required such as centralization, vertical communication, impersonal communication, and formal control to remove coordination relates issues.
The common challenges associated with agile RE activities are requirement elicitation, clarification, analysis, prioritization, documentation, and decision [31]. According to [121], agile software development process also integrates and assures the security of the software. The authors proposed a method to develop acceptably secure software providing the assurance of security, and the proposed method claims partially mitigation of associated threats. Rodriguez et al. [70] described some findings related to requirement engineering activities of agile system development. These findings are limitations in success of agile development. The intention of this project is to develop a product, and for development of product, some previous details are available. A list of existing and dropped features of initial product as well as a list of some new functionalities is presented. There are many issues in conventional processes, so according to author, they can be more critical for agile development processes.
7. Limitations
Though the agile approach has preformed the traditional software development approaches and has numerous benefits, it still has some drawbacks which are highlighted in this section as follows: No reusability of artifacts Lack of support for distributed environment Not suitable for development of safety critical systems Development of large projects with implementation of agile approach is more difficult [122]
The analysis of agile requirement engineering with respect to different agile frameworks or methodologies is not discussed comprehensively. We do not claim that the presented study is applicable universally on all development phases and methodologies. Our choice is limited in terms of agile requirement engineering, its methodologies, challenges being faced in agile requirement, and generic suggestions.
8. Conclusion and Future Work
Requirement engineering is critical and vital phase of software development life cycle. There are numerous challenges related to requirement engineering in agile development process. Though agile requirement engineering process provides benefits over traditional development cycles due to its features such as minimal documentation, quick implementation strategies, incorporating changing requirements, and frequent feedback from customers, it also has challenges and limitations. These challenges create adverse effects on product being developed such that quality and schedule of the product are affected. In this article, we have identified the issues and challenges of RE in ASD by reviewing recent state-of-the-art from the literature.
The formulated challenges and suggestions based on this study can be used to enhance the skills and abilities of the development companies during agile software development. Our research can serve as a baseline for the researchers who intend to work in this direction. It is very beneficial for research contributors who want to work in this study area which require practical results and studies with many approaches of agile such that crystal and extreme programming. Conflicts and challenges arise when the process is not defined. Hence, the conclusion drawn from this study is that the agile domain is still immature, and the need for establishing guidelines and standardization of the process is crucial to improve outcomes. Thus, there is need of guideline and standardization of process.
Data Availability
The data used to support this study are included within the manuscript.
Conflicts of Interest
The authors declare that they have no conflicts of interest.