The Naval Postgraduate School’s Department of Systems Engineering Approach to Mission Engineering Education through Capstone Projects

Abstract

This article presents an educational approach to applied capstone research projects using a mission engineering focus. It reviews recent advances in mission engineering within the Department of Defense and integrates that work into an approach for research within the Systems Engineering Department at the Naval Postgraduate School. A generalized sequence of System Definition, System Modeling, and System Analysis is presented as an executable sequence of activities to support analysis of operational missions within a student research project at Naval Postgraduate School (NPS). That approach is detailed and demonstrated through analysis of the integration of a long-range strike capability on a MH-60S helicopter. The article serves as a demonstration of an approach for producing operationally applicable results from student projects in the context of mission engineering. Specifically, it demonstrates that students can execute a systems engineering project that conducts system-level design with direct consideration of mission impacts at the system of systems level. Discussion of the benefits and limitations of this approach are discussed and suggestions for integrating mission engineering into capstone courses are provided.

1. Introduction

The Department of Systems Engineering at the Naval Postgraduate School (NPS) in Monterey, California has been educating graduate students on the topic of mission engineering through hands-on multi-term capstone projects for the last several years. While mission engineering is relatively new as a topic, the underlying techniques and methods are familiar to many systems engineers. This article defines mission engineering as an approach for simultaneously considering operations, acquisition, and integration and demonstrates the process by which the mission engineering approach is implemented and taught as part of the NPS Systems Engineering (SE) curriculum.

Educating students on mission engineering is of importance to the Department of Defense (DoD) and other large organizations to prepare the workforce for tackling large, complex missions that require multiple systems and assets working in concert. The demonstration of the student mission engineering capstone project presented in this paper shows how a platform such as the MH-60S helicopter is a multi-mission platform that is used to perform many different missions throughout the fleet. Understanding how altering the load-out of a MH-60S helicopter to achieve one mission objective can erode the ability of the MH-60S helicopter from meeting the objectives of other missions is an important and powerful learning opportunity for students.

Specific Contributions

This article contributes a description of the position that the Department of Systems Engineering at NPS has taken in educating graduate students on mission engineering through a capstone project. In particular, we state that mission engineering education should focus on simultaneous operational and system-level development to support warfighting mission effects, and system integration and system acquisition. Lessons learned and guidance is provided for other educators to rapidly adopt mission engineering into the curriculum.

2. Background and Related Research

The following section presents background and related research that defines mission engineering in the context of systems engineering and discusses several relevant related educational approaches.

2.1. Mission Engineering in the Context of Systems Engineering

There are multiple candidate definitions of mission engineering [1,2,3,4]. The National Aeronautics and Space Administration (NASA) has defined mission engineering in multiple forms [5,6,7], Wertz [8] presents a succinct summary, stating “mission engineering is the definition of mission parameters and refinement of mission parameters and requirements so as to meet the broad, often poorly defined, objective of a space mission in a timely manner at minimum time and risk”. That definition has been refined in recent years, first by Sousa-Poza [9] as an approach to coordinated the perspective of the mission owner, the operator, and the engineer and again by Dahmann and Gold [10,11,12] as “deliberate planning, analyzing, organizing, and integrating of current and emerging operational and system capabilities to achieve desired warfighting mission effects.” Similarly, ISO/IEC/IEEE 21839 [13] has emphasized the importance of consideration of the mission and deployed context in the development lifecycle. Specifically, increased focus is now given to not only the design of the system of interest, but also to the role of those individual systems in enabling the mission effects of an associated system of systems (SoS). In order to fully realize the benefits of mission engineering we propose an approach tailored for the use of modeling efforts throughout a traditionally implemented systems engineering process to support system development focused on mission execution. Specifically, per the mission engineering process presented by Hernandez, Karimova, and Nelson [3] and the Model-Based Systems Engineering (MBSE) process presented by MacCalman, Beery, and Paulo [14] mission engineering should be specifically focused on simultaneous operational and system-level development to support not only warfighting mission effects, but also system integration and system acquisition. Figure 1 present a high-level description of the mission engineering strategy as presented in Hernandez, Karimova, and Nelson [3].

2.2. The Naval Postgraduate School’s Systems Engineering Program

The NPS Department of Systems Engineering offers three resident master’s programs as well as two distance-learning master’s programs. These programs are open to both active duty U.S. Navy officers as well as officers from other services and DoD civilians. The programs focus not only on development of technical skills and expertise, but also focus on development of leadership and program management skills through operational use of those skills. In support of that goal, students are expected to produce an individual thesis or participate in an applied capstone research project that demonstrates the use of coursework concepts in an operational relevant subject.

Unlike many civilian institutions, the vast majority of NPS’s systems engineering students are mid-career professionals with deep DoD experience. NPS has no undergraduate programs and thus the Systems Engineering Department focuses exclusively on graduate education. While a small PhD program exists with roughly 20 students enrolled, the primary focus of the program is on master’s students.

2.3. Engineering Education

Several approaches are in use at NPS and other educational institutions to teach systems engineering. Texts such as Systems Engineering and Analysis [15], System Architecture: Strategy and Product Development for Complex Systems [16], The Art of Systems Architecting [17] serve as the foundation of many courses in systems engineering departments. In the related field of mechanical product design, The Mechanical Design Process [18] is used extensively. In the case of each textbook, the process of developing a system generally progresses as follows: identifying stakeholders, defining requirements, architecting the system, designing the system, prototyping, and testing the system prototypes, refining the system, manufacturing the system, fielding the system, and maintaining the system.

In the NPS systems engineering curriculum, some courses are homework-intensive and test-based while others are more focused on a term-long project or on lab sessions. All students complete a two or three term capstone project as part of a student team and under the guidance of multiple professors and instructors. The projects are often DoD-sponsored and generally have sponsors who are actively involved to shape the projects and participate in the in-progress reviews (IPRs). Before progressing from one stage of a capstone project to the next, capstone teams must successfully complete specific milestones and deliverables. This is similar to how some humanitarian engineering capstone and project courses at other universities are run [19]. However, one significant difference is that students are required to complete the entire project to receive a passing course grade. Dean and Van Bossuyt [19] advocated that a phase gate and sprint approach be used but with the ability for students to never move past a phase gate during the class if they are not yet ready to move forward. In the case of NPS capstone projects, this is not practical and would prevent students from having the full capstone experience. On rare occasions, a capstone project may be extended an additional term if a significant issue occurs although such instances are exceedingly rare.

Based on the NPS System Engineering Department’s current trajectory, it is anticipated that more courses will shift toward project-based curricula and portfolio-building activities will increase. Hands-on experiences with lab experiments, working with real systems that students may be familiar with from their prior work experience, and other improvements to the curricula are planned. Already, several courses such as the systems architecture course have shifted to have expanded hands-on components.

3. Overview of Capstone Course

The capstone teams at NPS are comprised of between 4–8 students, each of which completes a research project. Some students also complete a master’s thesis although not all programs require this. The projects cover a range of topics, from system component integration analysis to early stage operational concept development. The projects are associated with research sponsors who provide direction and guidance regarding the specific subject matter and are advised by a team of NPS SE faculty members, who guide the project execution. Traditionally, the projects either span the full spectrum of a systems engineering process similar to the Vee model as defined by Systems Engineering and Analysis [15] or conduct an in-depth analysis of system development in a segment of that process. Critically, to ensure relevancy and feasibility in a nine-month time frame the projects are focused on production or analysis of a specific system or system component. The projects generally span nine months and teams deliver three progress reviews to the faculty and sponsors. The projects that specifically implement a mission engineering approach expand the focus from analysis of a specific system or component to explicitly consider the impact that the design of that system may have on the mission effectiveness of the broader SoS in which the system will be deployed. (It should however be noted that focus in the capstone projects often remains on an individual system within a SoS and with a mission engineering context rather than on the larger SoS.) This necessitates a focus on three major areas: System Definition, System Design, and System Analysis. Figure 2 presents a notional capstone process consistent with best practices in SE that adheres to the general fundamentals of mission engineering.

Please note that the capstone process presented in Figure 2 is intentionally generic to allow for flexibility in subject matter. The capstone projects adhering to this mission engineering approach provide instructive feedback to the associated research sponsors in the area of System Definition, Modeling, and Analysis. Typically, those results are used to inform further studies that may be implemented in operational environments.

Mission Engineering Capstone Projects

While Figure 2 presents a general approach to analysis that has been used for multiple capstone projects at NPS, it has not been explicitly linked to the mission engineering concept. To achieve that linkage, recent capstones have applied the process from Figure 2 in parallel to each domain of mission engineering. Recall that the capstone teams at NPS are typically comprised of 4–8 students. This segmentation into operations, acquisition, and integration areas allows for parallel execution of complementary System Analysis. Portions of the team may focus their efforts on development of operationally focused models while other portions may focus on either acquisition or integration considerations. Recall that the first step in Figure 2 is Requirements Definition; consistent definition of system requirements across each of these domains is paramount for consistent modeling and analysis. The focus on consistency and coordination is a direct application of the coursework completed by NPS systems engineering students prior to commencement of the capstone project. Specifically, students have recently completed the following courses: Capability Engineering (providing a foundation in modeling and simulation as well as operationally focused system development), System Architecture & Design (providing a foundation for development of system requirements and functional/physical architecture fundamentals) and Systems Integration and Development (providing a holistic approach to system development emphasizing the relationship between each step in the engineering process).

Individual mission engineering-focused capstone projects often differ in the details of implementing the mission engineering process outlined above. However, they all adhere to the general process outlined in Figure 2. Students working with faculty tailor the mission engineering methodology to the specific project topic and need. While this is more labor-intensive than a cookie-cutter one-size-fits-all approach that is sometimes implemented in undergraduate capstone courses, NPS Department of Systems Engineering faculty generally feel that the benefits to the final project deliverables and the students are worth the extra overhead. This allows for future mission engineering capstone projects to more fully consider the larger SoS perspective than has been demonstrated in recent capstones.

4. Example Capstone Report

As a demonstration of the Mission Engineering Capstone Integration, the authors present an example capstone project by Broadfoot, et al. [20] that adheres to the process outlined in Figure 2. The study focused on the operational impact of the introduction of a rotary-wing aircraft equipped with a long-range standoff capability on the Navy’s Distributed Maritime Operations concept. Specifically, the project focused on appropriate operational employment of long-range engagement capable MH-60S helicopters in support of Anti-Surface Warfare (ASuW) missions.

4.1. MH-60S Capstone System Definition

The first phase of mission engineering-focused capstones is Requirements Definition and Architecture Definition, both of which comprise the general area of System Definition. Figure 3 presents the results of both of those process for the MH-60S project.

Please note that the Requirements Definition process resulted in two major focus areas: Processing of Target Data and Firing of a Long-Range Missile (LRM). These two major requirements are elaborated, and the resulting decomposition is used to inform the functional architecture development. Specifically, an iterative loop of Target Detection, Engagement, and Assessment for ASuW is defined as a SysML Activity Diagram. The key activities that enabled LRM use as a supporting system for DMO were detection of a target, firing of the LRM, and engagement of targets with the LRM. Accordingly, System Modeling was initiated with the focus on development of a model capable of representing each of those three primary activities.

4.2. MH-60S Capstone System Modeling

The second phase of mission engineering-focused capstones is System Modeling, defined by Baseline Modeling, Experimental Design, and Simulation Modeling. Figure 4 presents the results of both of those process for the MH-60S project.

There are several points of emphasis within Figure 4. First, the baseline system model is a direct application of the sequence of activities presented in the architecture development phase. Specifically, the model is focused on Target Detection, Engagement, and Assessment. System and range dependent probabilities of success for each potential target were defined and an initial model was created. After the creation of an initial model, an experimental design strategy was developed to ensure appropriate examination of the design space. Recall that the Requirements Definition process specifically called out processing of target data and engagement using an LRM as the two primary system requirements. To ensure adequate examination of those requirements within the model, eight system design variables were identified: Number of MH-60S, Number of LRM, LRM Maximum Range, LRM Minimum Range, LRM Probability of Hit, LRM Velocity, Probability of Target Detection, and LRM Ratio. Additionally, the model varied the number of enemy fast attack craft and fast inshore attack craft. After the full range of combinations was defined, the operational model was finalized.

4.3. MH-60S Capstone System Analysis

The third phase of mission engineering-focused capstones is System Analysis, comprised of Model Analysis, Dynamic Decision Support, and Reporting & Documentation. Figure 5 presents the results of both of those process for the MH-60S project.

During the Model Analysis phase, several major findings were developed, all focused on appropriate use of the LRM in support of ASuW missions. First, the capabilities of the LRM (maximum range, minimum range, velocity, and probability of hit), subject to the bounds established in the experimental design phase (Step 4 of the process as shown in Figure 5), were not statistically or operationally significant. This is a particularly valuable insight given that the factor that had the largest operational impact was the total number of LRMs. Succinctly, the team found that investment in larger numbers of less capable LRMs had a larger operational impact than investment in smaller numbers of more capable LRMs. Given the objective of the research was to inform development and use of the LRM in ASuW missions, the team felt this was an actionable recommendation. To provide specific context, the team developed tables and graphics that quantified the expected performance (in terms of friendly survivability and enemy attrition) that can be expected in a mission based on the number of LRMs employed by the friendly force.

5. Discussion

The NPS Department of Systems Engineering has received positive feedback both from DoD project sponsors and from students who have participated in mission engineering capstone projects. While the evidence is anecdotal, the faculty involved in mission engineering capstone projects feel that the definition of mission engineering being used and the approach to mission engineering being implemented by students is a success. Project sponsors have indicated their approval of the results of mission engineering capstone projects through repeated funding of new capstone projects. Students have provided feedback through course evaluation forms and informally that the mission engineering capstone projects were a highlight of their time at NPS and were useful in their post-NPS positions.

One limitation of the NPS Department of Systems Engineering approach to mission engineering capstones is that the students are often not formally informed that they are conducting mission engineering. Due to the emerging nature of mission engineering as a topic of interest to DoD and systems engineers, NPS has been teaching mission engineering for several years without describing it as such. In future mission engineering capstone project courses, the faculty is planning to more fully and explicitly brief students on mission engineering.

Another limitation of the approach to mission engineering presented above is that students are unable to complete the entire cycle of mission engineering as described by Dahmann [12]. Within the confines of a two- or three-term capstone project, prototyping and/or experimenting outside of a computer simulation is often impossible. For instance in the case of the MH60s helicopter capstone project described above, while several of the students were helicopter pilots, students were unable to test load-out configurations in the real-world.

The short duration of student capstone projects also adversely impacts the ability of a larger SoS mission engineering analysis to be conducted. While aspects of the SoS in which a system of interest is operating are considered, such as in the case of the MH-60S project where other ships and helicopter assets were considered in the simulations [20], the larger vision of mission engineering looking across the entire SoS has to date not been fully achieved. However, the flexibility of the implementation of mission engineering capstones in the NPS Systems Engineering Department allows for future capstone projects to have expanded SoS scope.

Much like the discipline of systems engineering itself, mission engineering is a rapidly evolving field without a strong, agreed upon definition and framework. While the mission engineering definition and framework presented above is the authors’ interpretation of mission engineering, others may categorize mission engineering as a permutation of systems engineering. SoS engineering, model based systems engineering, and other related topics also may lay claim to part or all the processes involved in mission engineering. It is the authors’ expectation that how mission engineering is defined and how mission engineering is performed will continue to rapidly evolve over the next several years.

The outcome of mission engineering capstone projects in the NPS Department of Systems Engineering is generally a report that details the analysis, conclusions, and recommendations developed by the student teams in conjunction with the faculty. Often, simulation models and other work products are also delivered in a final work product package. Several IPR briefings and a final project briefing are also produced. This information is then conveyed to the project sponsors and is archived in the NPS library where the public may view the documents.

5.1. Outcomes to Date

An estimated 30–40% of capstone projects in the NPS Department of Systems Engineering completed over the last five years have been mission engineering-focused. This translates to roughly 5–7 mission engineering capstone projects and approximately 30 students participating per year. For several examples of mission engineering capstone projects, see: [21,22,23,24,25,26,27,28,29,30,31]. The vast majority of mission engineering capstone projects at NPS have DoD sponsors funding the efforts. Interest in mission engineering capstone projects has increased among project sponsors in the last 2–3 years and is expected to continue to increase in the future.

5.2. How to Implement Mission Engineering in a Capstone Project Class

While no one-size-fits-all approach exists to implement mission engineering in capstone courses, the authors have the following suggestions:

• Identify strong mission engineering capstone projects for students to work on. Allowing students to define projects sometimes does not deliver desired results.
• Explicitly instruct students on the nature of their capstone projects. Provide literature and lectures on mission engineering. Specifically, ensure that students working on mission engineer capstones are focused on analysis of trades at the system level that may impact mission performance at the SoS level.
• If at all possible, ensure that there is a project sponsor who wishes to be actively involved.
• Require weekly updates from the student teams to describe their progress, pain points, areas they want help/guidance on from the faculty advisors, and other related information.
• Implement quarterly IPR presentations with defined deliverables.
• Provide standardized IPR presentation and final report templates to the student teams.

In addition to the above suggestions, we believe more explicitly addressing mission engineering in other classes may be useful. For instance, the system architecture course (SE 4150), the systems engineering introduction course (SE 3100), the system suitability course (SE 3202), and the combat systems integration course (SE 4115) in the Naval Postgraduate School’s Systems Engineering Department curriculum may all be good candidates for integration of mission engineering modules. The authors teach these and other courses in the curriculum where natural fits exist to explicitly elicit discussion and instruction on mission engineering. Already in several of the above-mentioned courses, elements of mission engineering are discussed in the context of SoS engineering, design reference missions, and maintenance logistics. More explicitly weaving mission engineering into coursework outside of the capstone class may be beneficial to both students who conduct mission engineering-focused capstone projects and students who conduct system-level or SoS-focused capstone projects.

6. Future Work

The authors plan to develop mission engineering lectures to deliver to students at the start of their capstone experience. A dedicated mission engineering class may be added to the curriculum as an elective in the future. The authors also intend to reach out to other institutions and actively solicit interested faculty at other institutions to contact them to build a mission engineering education community to improve how mission engineering is taught.

7. Conclusions

This article presented an educational approach and example of an applied capstone research project using a mission engineering focus. The mission engineering capstone projects that NPS Department of Systems Engineering students have been completing over the last five years have been well-received by students and project sponsors. A generalized sequence diagram and description of how NPS faculty have run mission engineering capstone projects is provided. Discussion of the strengths and weaknesses of the approach are provided and specific suggestions to have a successful implementation of mission engineering into capstone projects is given. A student capstone project of a MH-60S helicopter load-out is shown as a demonstration of a result from a mission engineering capstone project.