4.3 HSI in the System Development and Demonstration Phase (Pre Milestone C)

Overview

As described in DoDI 5000.2, the purpose of the System Development and Demonstration phase is to develop a system, reduce program risk, ensure operational supportability, design for prodHCIbility, ensure affordability, ensure protection of Critical Program Information, and demonstrate system integration, interoperability, and utility. Discovery and development are aided by the use of simulation-based acquisition and test and evaluation and guided by a system acquisition strategy and test and evaluation master plan (TEMP). System modeling, simulation, test, and evaluation activities shall be integrated into an efficient continuum planned and executed by a test and evaluation integrated product team (T&E IPT).

This continuum shall feature coordinated test events, access to all test data by all involved Agencies, and independent evaluation of test results by involved Agencies. Modeling, simulation, and development test shall be under the direct responsibility of the PM or a designated test agency. All results of early operational assessments shall be reported to the Service Chief by the appropriate operational test activity and used by the MDA in support of decisions. The independent planning, execution, and evaluation of dedicated Initial Operational Test and Evaluation (IOT&E), as required by law, and Follow-on Operational Test and Evaluation (FOT&E), if required, shall be the responsibility of the appropriate operational test activity (OTA).

This phase can be entered either directly out of technology opportunity and user need activities or from Concept Exploration. The actual entry point depends on the maturity of the technologies, validated requirements (including urgency of need), and affordability. The MDA shall determine the appropriate entrance point, which shall be Milestone B. There shall be only one Milestone B per program, or evolutionary block.

Entrance Criteria

Entrance into System Development and Demonstration is dependent on three things: technology (including software) maturity, validated requirements, and funding. Unless some other factor is overriding in its impact, the maturity of the technology will determine the path to be followed. Programs that enter the process at Milestone B shall have a system architecture and an operational architecture for their relevant mission area.

Technology is developed in S&T or procured from industry. Technology must have been demonstrated in a relevant environment (reference (c) for a discussion of technology maturity) or, preferably, in an operational environment (using the transition mechanisms) to be considered mature enough to use for product development in systems integration. If technology is not mature, the DoD Component shall use alternative technology that is mature and that can meet the user's needs. The determination of technology maturity is made by the DoD Component S&T Executive, with review of the determination for MDAPs by the DUSD(S&T). If the DUSD(S&T) does not concur with the determination, the DUSD(S&T) will direct an independent assessment. To promote increased consideration of technological issues early in the development process, the MDA shall, at each acquisition program decision, consider any position paper prepared by a Defense research facility on a technological issue relating to the major system being reviewed; and any technological assessment made by a Defense research facility . A defense research facility is a DoD facility that performs or contracts for the performance of basic research or applied research known as exploratory development.

Prior to entering System Development and Demonstration, there shall be an ORD validated by the requirements authority. The ORD contains operational performance requirements and addresses cost for a proposed concept or system. Time-phased ORDs must be validated by the requirements authority prior to program approval. If a mature technology, non-developmental item, or commercial item is being considered for transition to an acquisition program at Milestone B or C, it must have a validated ORD prior to being approved as an acquisition program.

The affordability determination is made in the process of addressing cost as a military requirement in the requirements process and included in each ORD, beginning with the acquisition cost but using life-cycle cost or total ownership cost where available and approved. Transition into System Development and Demonstration also requires full funding (i.e., inclusion in the budget and out-year program of the funding for all current and future efforts necessary to carry out the acquisition strategy), which shall be programmed when a system concept and design have been selected, a PM has been assigned, an ORD has been approved, and system-level development is ready to begin.

Overview of System Development and Demonstration Phase HSI Activities

Relationship to the systems acquisition process

Substeps/Activities/Guidelines

Step 8 Develop Human-Centered Design Concepts

Overview: This step is concerned with the development of HSI design concepts in the context of alternative system design concepts. HSI design concepts include concepts for human-machine interfaces, user-computer interface (HCI), workstations, procedures/documentation/decision aiding, workspace/facility, maintainability design, safety & health hazard avoidance design, and training system design.

Relationship to the System Acquisition Process

The user or user's representative will interact with the program office and the DoD Component operational test and evaluation activity during this phase to assist in the evaluation of design alternatives, to support in developing operational assessments of any prototypes built, and to identify opportunities for cost-schedule-performance trade-offs among the various design approaches. The user or user's representative will update and expand the Operational Requirements Document to reflect system definition and prototype experience during the system development and demonstration phase. Minimum acceptable requirements for key parameters in the Operational Requirements Document will be incorporated in the Development Baseline as thresholds. Performance objectives and thresholds in the Development Baseline will be reviewed by the validation authority prior to Milestone II to confirm that they provide an operational capability that satisfies the mission need. For operational constraints relevant to the preferred concept(s), an initial list of critical system characteristics with proposed thresholds and objectives will be identified in the Operational Requirements Document. The cost and risk of providing the proposed critical system characteristics will be assessed during this Phase. Alternative approaches for providing these capabilities will be identified and addressed. The user or user's representative will participate in the selection and evaluation of these alternatives. Cost-schedule-performance trade-offs will be considered in preparing the proposed final list of critical system characteristics.

Inputs

Human-system interface requirements

Lessons learned from existing systems

Outputs

Design concepts for space and machinery arrangements

Design concepts for human-machine interface (HMI)

Design concepts for maintenance

SUBSTEPS/ACTIVITIES/GUIDELINES

8.1 Identify Human Interface Requirements

Overview: Human interfaces represent the multi-faceted approaches for designing the system in light of the capabilities and limitations of the humans who will perform assigned roles in the operation, maintenance, support, and management of the system.

This step addresses the identification of human interface requirements for each alternative design concept. Specific classes of human interfaces include: functional interfaces, informational interfaces, environmental interfaces, operational interfaces, organizational interfaces, cooperational interfaces, cognitive interfaces, and physical interfaces.

When requirements associated with these interfaces have been described for each alternative design concept, a clearer picture will evolve of what specific design issues must be addressed for each concept in order to accommodate the capabilities and limitations of humans.

Relationship to the System Acquisition Process

The Operational Requirements Document (ORD) identifies minimum acceptable performance requirements to satisfy the operational need. It also includes performance objectives that would provide operationally meaningful increases in capability.

The ORD produced at Milestone I describes each concept proposed for continued evaluation in Phase I, Demonstration and Validation. During Phase I the ORD is updated and expanded to include thresholds and objectives for more detailed and refined performance capabilities and characteristics based on the results of trsadeoff studies and testing conducted in Phase I. In developing capabilities and characteristics from a HSI perspective, the initial step is to clearly establish the roles, responsibilities, and requirements of the human in each design alternative. This is accomplished through the identification and analysis of requirements associated with each category of human interfaces for each alternative.

Inputs

Results of the role-of-human determination for each alternative from Step 8;

Results of functional, task, and performance requirements analyses for each alternative.

System requirements from the MNS and ORD;

ORD description of each design alternative.

Outputs

Human interface requirements for each alternative.

Substeps/Activities/Guidelines

8.1.1 Identify functional interface requirements ­

8.1.1.1 Identify role of the human requirements. Conduct allocation of functions and identify requirements for the role of the human in manual functions, automated functions, and interactive functions

8.1.1.2 identify constraints on manning levels - identify operational position requirements maintenance position requirements and supervisory position requirements

8.1.1.3 Conduct task analysis

8.1.1.4 Identify skill requirements by position ­ review and classify skill requirements from task analysis

8.1.1.5 Identify training requirements - identify need for new training and identify training system requirements

8.1.1.6 Identify training device design requirements - information reception media, skills acquisition media, fidelity levels, and display formats

8.1.2 Identify informational interface requirements

8.1.2.1 Compile information requirements by position

8.1.2.2 Classify information requirements

8.1.2.3 Priortize information requirements

8.1.2.4 Identify information display design requirements

8.1.2.5 Identify information display format requirements

8.1.2.6 Identify information display content requirements

8.1.3 Identify environmental interface requirements

8.1.3.1 Identify physical environmental interface requirements - expected temperature range, noise environment, vibration levels, terrain, and lighting levels

8.1.3.2 Identify psychological environmental interface requirements - need for sustained operations, potential for stress, and potential for disorientation

8.1.3.3 Identify tactical environmental interface requirements - need to perform wearing survival gear, and tactical situation factors which affect workload

8.1.4 Identify operational interface requirements

8.1.4.1 Identify requirements for procedures

8.1.4.2 Identify requirements for documentation

8.1.4.3 Identify requirements for job aids

8.1.4.4 Identify requirements for on-line training/tutoring

8.1.5 Identify organizational interface requirements

8.1.5.1 Identify job requirements

8.1.5.2 Identify management requirements

8.1.5.3 Identify command structure requirements

8.1.6 Identify cooperational interface requirements

8.1.6.1 Identify team activities

8.1.6.2 Identify communication requirements

8.1.6.3 Identify personnel interaction requirements

8.1.6.4 identify interoperability requirements

8.1.7 Identify cognitive interface requirements

8.1.7.1 Identify decision requirements

8.1.7.2 Identify situation awareness requirements

8.1.7.3 Identify problem solving requirements

8.1.7.4 Identify requirements for establishing mental models

8.1.8 Identify physical interface requirements

8.1.8.1 Identify workstation design requirements

8.1.8.2 Identify workspace design requirements

8.1.8.3 Identify facility design requirements

Step 8.2 Identify Requirements for HMI Design for Operability and Interoperability

Overview: The aspect of a system which addresses design to support human interaction with information and system control is designated as the human-machine interface (HMI). HMI is defined as encompassing all elements of a weapon system or an information management system with which the human user must interact. System elements which impact HMI include workstations, I/O hardware, software, data bases, networks, computation systems, peripheral devices, communications systems, and software engineering environments. HMI can be conceptualized as including displays, displayed information, display characteristics, display formats, integration of displays, labels, instructions, alarms, symbology and graphics, decision aids, decision support systems, input devices, data designation and manipulation devices, user interface language, expert systems and knowledge-based systems, interactive user-computer dialogues, command language, command modes, controls and controllers, control systems, control and display arrangements, communications, workspace layout, workspace environment, help features, embedded training, intelligent tutoring systems, and procedures.

A critical need exists for standardization of workstation human-machine interfaces. Before such standardization can occur, workstation HMI operability issues must be considered. HMI operability and interoperability objectives include the following:

Relationship to the System Acquisition Process

The development of detailed HMI design for operability will focus on prototyping HMI concepts to assess and reduce the risks associated with integrating available and emerging HSI technologies into a system design approach to satisfy a validated mission need. Test and evaluation of prototypes will confirm the feasibility of specific design approaches relative to its ability to satisfy te mission need and achieve minimum acceptable operational performance requirements within affordability constraints. Prototyping will be used to assess cost and performance tradeoffs.

DoD 5000.2 directs that in test and evaluation planning, all system components should be addressed including human interfaces.

Inputs

Results of Step 3.0 Develop HSI Design and Readiness Concepts

Outputs

Concepts and criteria for HMI design for operability

SUBSTEPS/ACTIVITIES/GUIDELINES

8.2.1 Develop operability human-machine interface (HMI) design concepts

8.2.1.1 identify human-machine interface approaches implemented in predecessor or baseline systems, and identify problems with each approach

8.2.1.2 describe alternate human-machine interface concepts for specific missions, conditions and task sequences ­ develop concepts for: displays, controls, consoles, workstations, feedback approaches, information integration, equipment arrangements, workspace layout, job performance aids, decision support systems, component labeling, display coding.

8.2.1.3 conduct studies and simulations to develop alternative concepts and to evaluate the effectiveness of human-machine interface concepts

8.2.1.4 Conduct tradeoffs - describe design alternatives; define tradeoff criteria; complete tradeoffs

8.2.1.5 Describe the design concepts for operability , including concepts for:

8.2.2 analyze requirements, concepts and criteria for W/S design for operability

8.2.2.1 design of human-machine interfaces to reduce errors

8.2.2.2 design of human-machine interfaces to reduce workloads

8.2.2.3 design of human-machine interfaces to simplify tasks

8.2.2.4 design of procedures

8.2.2.1 design of communications

8.2.3 analyze requirements, concepts and criteria for W/S design for fightability

8.2.3.1 design of battle management displays

8.2.3.2 design of battle assessment techniques

8.2.3.3 design of decision aids - a decision aid or decision support system is an aid or tool, usually computerized, which assists the decision maker in the processes of formulating, implementing and verifying a decision. The characteristics of human expertise that surpass the capabilities of automation include the human capacity for creativity, adaptability, and the facility to incorporate experience, a broad focus, logical reasoning, and common-sense knowledge. The goal of a decision support system should be to capitalize on the strengths of the human along with the advantages of automation. The functions to be performed by decision support systems include the following:

8.2.4 Identify requirements for Human-Centered Automation Design - Ship manpower reduction and human performance and safety improvement concepts are being developed through the application of human systems integration (HSI). An element of HSI concerned with the design of human machine interfaces to support the performance of humans in highly automated systems is designated Human-Centered Automation (HCA).

HCA is an expansion of the human engineering element of HSI which is directed specifically to design of tactical systems, protocols, procedures, information systems, and human machine interfaces to support human-automation interaction.

HCA is the systems engineering approach to automation focused on the roles and requirements of humans interacting, cooperating, coordinating, and collaborating with automated processes.

HCA is concerned not only with human requirements in automated processes, but also with requirements for automation to support human performance, such as with decision support systems and operator's associates.

8.2.4.1 Application of HCA methods and data will lead to requirements and techniques for:

8.2.4.2 The major issues with automation and its impact on human performance include the relationships among automation, manning and workload; the degree of trust that a human operator has in automation; and the interrelationships among situation awareness, erroneous expectancy, and stress.

8.2.4.3 The steps of applying HCA in system design are as follows:

1) Allocate functions - The initial step is to allocate functions to identify which system functions should be performed by humans or should require interaction between human and automation. The allocation of functions will result in a decision concerning how the function will be performed along a continuum from totally automated to totally manual. The decision points on this continuum are as follows:

2) Identify human and machine roles and requirements - This step will entail identifying human and machine roles for system functions, and define requirements associated with performance of these roles. With the completely automated allocation strategy the only human role is to monitor the automated system, and possibly intervene in the case of automated system failure. In the situation where the allocation is to automation with human supervision, human involvement includes supervising the automated process, monitoring, planning, adjusting, and reporting. The number of potential human roles increase significantly with the two manual allocation modes.

3) conduct task analysis - Having allocated functions and assigned roles of humans and machines, the next step is to conduct a task analysis to identify:

The task analysis is the critical step for determining where interactions between human and automation are required, and how they can be carried out. As requirements for human performance of tasks are identified (information, performance, decision, and support requirements) needs are also established for automation support in: maintaining situation awareness, maintaining tactical perspective, making decisions and tactical judgments, determining causes, selecting actions, and establishing priorities.

In addition to identifying requirements, the task analysis will focus on the potential for human error with alternate approaches to function automation and human-automation interaction.

4) develop automation concepts and associated human performance requirements This step involves developing automation requirements and concepts for ensuring effective interaction and coordination between human performance and automation. Automated system concepts will be developed for such activities as providing the operator with current and complete awareness of the tactical situation, providing means to accomplish monitoring and verification, enabling human intervention into automated processes, supporting planning and supervision, providing support for decision making, problem solving, and information integration, and enabling the operator to reassign roles and responsibilities between human and machine in real time.

5) Identify technology requirements - Technologies needed to support automation concepts include alternative software tools and techniques. A classification of decision support systems implementations to include the following:

intelligent tutoring system An advanced training technology that combines artificial intelligence, modeling, and cognitive psychology to develop expertise. It provides training for novices to experts within specific knowledge domains and also can provide certification and refresher training for all levels of expertise.

situation awareness aid which collects data from all available sources to characterize a model of what is happening in the external world of interest to the operator, what to expect, what actions are required, what additional information is needed, what's important, and how much time is available for completing the model.

real-time simulation for investigating the potential outcome of planned activities, assessing alternate diagnoses, and rehearsing action strategies prior to implementation.

cooperative, collaborative decision making wherein humans interact with other humans and with intelligent machines which serve to enhance or augment the operator's decision-making capabilities This class of models is called cooperative decision making emphasizing the importance of the cooperation between the multiple intelligent agents, both human and machine. The machine contribution to cooperative decision making typically includes what are termed advisory strategies which include advice formulated by the machine concerning what action the human should pursue and how much time is available.

integration of data fusion with decision support - data fusion is the collection and representation of data from a number of sources in order to give the operator the best current view of the situation. Decision support concentrates on the evaluation of alternatives. In any decision augmentation system, these two functions are closely coupled and the data fusion system is a major source of input to the situation assessment module of the decision augmentation system. Because the development of a decision augmentation system is dependent on the development of a data fusion system, there should be an integration of these two systems early in the development process.

operator's associate (or pilot, commander, evaluator, or maintainer) this class of decision support systems incorporates many of the features of categories described above to enable the intelligent machine act as an aide to the human in the performance of missions, functions and tasks.

6) assess automation and HMI design concepts through modeling and simulation - In this step the effectiveness of alternative configurations of machine and human roles and responsibilities can be evaluated in representative mission scenarios using modeling and simulation (M&S). The HSI M&S approaches of major relevance here are two: (a) task network simulation to determine the effectiveness of task sequence performance with time constraints; and (b) human-in-the-loop simulation to assess human performance with alternate levels of automation control and support.

7) Apply human-centered design methods and data - Having defined the roles and requirements of humans in the conduct of mission scenarios, and the techniques of human-automation interaction to provide needed levels of human performance, the final step is to develop design requirements for human-machine interfaces (HMI). The objectives of human-centered design include: develop design concepts which foster and support the interaction between human and automation; develop design concepts which will enhance human performance and safety; develop design concepts which reduce the incidence and impact of human error; and develop design criteria to support design concepts. Human-centered design is concerned with defining design requirements and concepts for HMI, to include all interfaces between humans and other system elements, including hardware, software, information, environments, organizations, procedures, and other humans. HMI can be defined in terms of the following specific types of interfaces: functional, informational, operational, organizational, environmental, cooperational, cognitive, and physical.

8) Model human-automation interactions In the human-centered design process the pivotal step is the development of models of the interactions and transactions between human and automation. This modeling effort will identify

Based on human-automation models, concepts for HMI and automation will be developed and assessed through modeling and simulation, human error analysis, and tradeoffs. Error analysis will assess error potential for automation and HMI concepts. HMI design criteria will then be developed.

8.2.5 Identify HSI requirements for interoperability, to include:

8.3 Identify Requirements for Human-Machine Interface Design for Usability

Overview: The development of detailed human computer interface (HCI ) design for usability will focus on prototyping HCI concepts to assess and reduce the risks associated with integrating available and emerging HSI technologies into a system design approach to satisfy a validated mission need. Test and evaluation of prototypes will confirm the feasibility of specific design approaches relative to its ability to satisfy te mission need and achieve minimum acceptable operational performance requirements within affordability constraints. Prototyping will be used to assess cost and performance tradeoffs.

The DoD Joint Technical Architecture (Ve3rsion 4.0, April, 2001) states that the objective of system design is to ensure system reliability and effectiveness. To achieve this objective, the human must be able to effectively interact with the system. Humans interact with automated systems using the HCI. The HCI includes the appearance and behavior of the interface, physical interaction devices, graphical interaction objects, and other human-computer interaction

methods. A good HCI is both easy to use and appropriate to the operational environment. It exhibits a combination of user-oriented characteristics such as intuitive operation, ease and retention of learning, facilitation of user task performance, and consistency with user expectations. The need to learn the appearance and behavior of different HCIs used by different applications and systems increases both the training burden and the probability of operator error. What is required are interfaces that exhibit a consistent appearance and behavior both within and across applications and systems.

DoD 5000.2 directs that in test and evaluation planning, all system components should be addressed including human interfaces.

Relationship to the System Acquisition Process

In the determination of requirements for HMI design for usability, the emphasis is on human-computer interface (HCI). Major issues in HCI design include: techniques for data access; design of data retrievaldialogues; requirements for command modes; data entry device design; display screens and display formats; flexibility in page composition and data compilation; dedicated versus multi-function displays; data designation design requirements; on-line help mode design requirements; display transparency-navigation requirements; and direction, prompting, cueing design requirements.

Inputs

Standards identified in the DoD Joint Technical Architecture (2001) to address HCI and elements of HCI include the following:

ISO 13407:1999(E) Human-centred design processes for interactive systems, June 99.

MIL-STD-2525B, Common Warfighting Symbology, 30 January 1999

MIL-STD-1787C (USAF), Aircraft Display Symbology

The Windows Interface Guidelines for Software Design,le Microsoft Press, 1995.

MIL-PRF-89045, DoD Performance Specification Geospatial Symbols for Digital Displays (GeoSym), 20 February 1998

The DoD Human-Computer Interface Style Guide, dated 30 April 1996, is part of the TAFIM. The TAFIM has been officially withdrawn. The draft DoD HCI Style Guide, Version 4.0, dated 21 December 2000, is a stand-alone updated version, which will officially replace the TAFIM DoD HCI Style Guide when it is finally approved. The following standard is emerging: Draft DoD Human-Computer Interface Style Guide, Version 4.0, 21 December 2000.

Outputs

HCI Design concepts & criteria

SUBSTEPS/ACTIVITIES/GUIDELINES

8.3.1 Develop Human-Computer Interface (HCI) design concepts

8.3.1.1 review operability/maintainability concepts

8.3.1.2 identify requirements for human-computer interface

8.3.1.3 identify tasks requiring a human-computer interface

8.3.1.4 identify requirements for specific HCI features

8.3.1.1 Develop HCI Concepts

8.3.2 Develop workstation design requirements

8.3.2.1 Review descriptions of the workstation requirements

8.3.2.2 Identify requirements to develop new workstations, use NDI, or improve existing workstations

8.3.2.3 Identify specific requirements for human engineering input to workstation design

8.3.2.4 Define Workstation ob-jectives and functions

8.3.2.5 Conduct Function Analysis - identify functions and construct functional flow block diagrams

8.3.2.6 Allocate Functions - provide cognitive support in response to specific questions, such as:

8.3.2.7 Conduct cognitive task analysis ­ a process by which a cognitive task or cognitive aspects of tasks are described and examined, and characteristics are identified (e.g., workload, information requirements, control re-quirements). The purpose of conducting a cognitive task analysis is to: (1) identify the actions the equipment and user must take in order to accomplish usercognitive functions and, (2) provide a ba-sis in which the tasks identified can be analyzed to determine what information is required in order to support these tasks. The specifics of conducting a cognitive task analysis are presented in Step 12.2.

8.3.2.8 Select Display Elements - associ-ate each required item of information identified during the task analysis with some element in a picture in order that it may be displayed, identify display elements other than icons, and develop icon design concepts

8.3.2.9 Construct Display Pages

8.3.2.10 Conduct paper and pencil walkthrough

Using checklists developed from design guideline hand-books - identify any major human factors deficiencies that might compromise understandability or effectiveness of the proposed displays.

Walkthroughs addressing understandability deal with content density, content integration, format orientation, and cognitive fidelity.

8.3.2.11 Define Display Relation-ships

8.3.2.14 Conduct Experiments - production software and ac-companying documentation should be complete, and a series of observations are made under controlled conditions for the purpose of testing a hypothesis.

8.3.3 analyze requirements, concepts and criteria for W/S design for usability

8.3.3.1 design of user-computer interfaces

8.3.3.2 design of human-machine interfaces with expert systems

8.3.3.3 design of procedures and documentation

8.4 Identify Requirements for Human-Machine Interface Design for Maintainability

Overview: Maintainability design requirements include information requirements, design for accessibility, equipment arrangement to facilitate maintenance, procedures, troubleshooting diagnostics and decisions, skill levels and maintenance training, equipment design for maintainability, allocation of maintenance responsibility to human or machine, equipment installation requirements, requirements for special tools and support equipment, job aid requirements, communication requirements, facility design requirements, and safety design requirements

Relationship to the System Acquisition Process

DoD 5000.2 states that maintainability objectives will address servicing, preventive (scheduled maintenance), corrective (unscheduled maintenance), and battle damage repair in terms of allowable downtime or turnaround time, required manpower, skill levels, special tools and test equipment, and diagnostic capabilities.

A firm objective will be established at Milestone B for each applicable reliability and maintainability parameter. Reliability and maintainability growth will be assessed during Phase II to ensure that objectives are met before the production decision.

Inputs

Maintainer requirements; lessons learned data from predecessor systems

Outputs

Maintainability requirements, concepts and criteria as they impact human performance, safety, and readiness.

SUBSTEPS/ACTIVITIES/GUIDELINES

8.4.1 Application of HSI to the design for maintainability

Specific goals of HSI application to system design for maintainability include the following:

a) Reduce the need for maintenance. The need for maintenance is reduced through employment of high reliability equipment, and attention to human reliability. The GAO has determined that half of the maintenance requirements in military systems result from errors on the part of maintainers and operators. A major goal of HSI is the enhancement of human reliability through reduction of the incidence and impact of human error , and making systems error tolerant.

b) Reduce the time to repair. Time to repair, time to reconfigure system components, and time to conduct tests will be reduced through a more usable design, including use of troubleshooting practices which take into account human decision-making capabilities, improved maintenance access, simplified design concepts, improved alarms and annunciators, and improved procedures.

c) Reduce the incidence and impact of maintainer human error. As indicated above, human error is the major cause for system failures. HSI methods for reducing human errors include: the imposition of human factors engineering design standards; reliance on test and evaluation procedures, as interviewing subject matter experts, examination of work samples, observation of task sequences, and use of simulation; and investigation of critical incidents to understand the dynamics and etiology of human error.

d) Reduce maintainer workload and manning levels required. for maintenance Cognitive workloads are reduced through improved diagnostics, procedures, and decision aids. Maintenance manning requirements are reduced through imposition of human factors engineering design standards, maintenance task simplification, improved design of human-machine interfaces, improved maintenance information handling, improved automated test and diagnosis, and maintenance job design/job aids to reduce the need for multi-person maintenance tasks. Maintenance manning requirements are also reduced by providing maintenance personnel with expert advise and decision aiding to reduce the number and scope of maintenance skills required on board. This expert advice/aiding can be provided in one or both of two approaches: by means of on-board intelligent maintainer associate systems or expert systems to support maintainer decision processes, and by means of tele-maintenance wherein maintenance advice is provided to the on-board maintainer via communications links with experts at a remote location.

e) Reduce maintainer skill requirements and training burden. Skill/training reductions result from design simplification, procedures improvement, application of advanced instructional technology, and use of decision aids. Skill requirements will also be reduced by providing maintenance personnel with expert advise and decision aiding to reduce the number and scope of maintenance skills required on board. As described above, this expert advice/aiding can be provided through an on-board maintainer associate, or via tele-maintenance.

f) Improve the design for maintenance access. Methods include imposition of human factors engineering workspace design standards, and development of models and mockups to assess the accessability of components and subassemblies for removal/replacement or in-situ maintenance. A major issue in accessibility is the physical anthropometry of the maintenamce personnel, which can range from the 5th percentile female to the 95th percentile male.

g) Improve maintenance procedures. Application of HSI will improve specific features of procedures, including accessibility, content, and organization, by ensuring that the procedure is complete, correct, clear, concise, current, consistent, and compatible with the reading/language/cognitive skill levels of the intended users.

h) Enhance maintainer safety and health, through hazard identification, design to eliminate or control safety hazards, and design of jobs to reduce the incidence of health hazards. Since safety is a major concern in achieving a reduced ship manning concept, the HSI program will emphasize the techniques for maintaining crew safety throughout all maintenance evolutions.

i) Increase maintainer productivity. by ensuring that equipment is usable, that workloads are reasonable, that stress associated with the job is reduced, that the worker is safe, that attention has been focussed on the role of personnel versus automation in the conduct of maintenance tasks, and that the design for maintainability will enable workers to work faster with a heightened level of job satisfaction and personnel safety.

j) Enhance system affordability and reduce life cycle costs by reducing costly errors and accidents, reducing system downtime by reducing time to repair, reducing training time through task simplification and use of on-line decision aiding, and reducing the numbers and skills of personnel required.

8.4.2 Develop maintainability design concepts

8.4.2.1 Define Maintainability Design Requirements

8.4.2.2 Develop Maintainability Design Concepts

8.4.2.3 Develop Maintainability Design Criteria

8.4.3 analyze requirements, concepts and criteria for the workstation design for maintainability

8.4.3.1 design for accessibility

8.4.3.2 design of troubleshooting diagnostics

8.4.3.3 design of maintenance human-machine interfaces

8.5 Identify Requirements for Human-Machine Interface Design for Supportability

Overview

Dod Instruction 5000.2, Part 7, Section A "Integrated Logistics Support" demands that an effective integrated logistics support (ILS) effort be established within each program office. ILS shall be managed as a disciplined, unified, iterative approach to the management and technical activities necessary to:

1) develop support requirements that are related consistently to readiness objectives, to design, and to each other;

2) effectively integrate support considerations into the system and equipment design;

3) identify the most cost-effective approach for supporting the system when it is fielded;

4) ensure that the required support structure elements are developed and acquired.

The acquisition strategy will identify resource requirements and explicit planning to avhieve these objectives. The acquisition strategy will emphasize: early identification of support and supportability requirements including any planned use of warranties; evaluation of alternate support concepts and techniques to reduce cost and support risk; and identification of test articles to conduct reliability, maintainability, and logistics supportability test and evaluation.

Integrated logistics support efforts shall encompass the ten elements identified below:

Preliminary peacetime and wartime readiness objectives and thresholds will be established by Milestone A, Concept Demonstration Approval, and final objectives and thresholds will be established by Milestone II, Development Approval. The acquisition strategy will dentify resource requirements and include explicit planning for achieving these objectives. The acquisition strategy will emphasize: early identification of support and supportability requirements including any planned use of warranties; evaluation of alternative support concepts and techniques to minimize cost and support risks; identification of test articles needed to conduct reliability, maintainability, and logistics supportability test and evaluation; and contractor incentives for timely attainment of support related design objectives.

The management approach, decisions, and plans associated with logistics planning efforts will be documented in an Integrated Logistics Support Plan (ILSP). This plan will be the basis for coordinating logistics planning efforts and ensuring that each one of the integrated logistics support elements is addressed and integrated with the other elements throughout the program; and

The Integrated Logistics Support Plan will be prepared in close coordination with the Computer Resources Life-Cycle Management Plan and will directly reference that plan. For computer resources or software that will be transferred to logistics organizations for maintenance or modification, areas to be addressed for software support will include special manpower skills, facilities, software tools, and special purpose computer requirements.

Integrated logistics support planning must be focused at the level at which support resources must be integrated to affect maintenance (i.e., the level at which specific repair or maintenance will occur). This is usually at the subsystem or below. The Integrated Logistics Support Plan will reflect this focus.

A tailored logistics support analysis (LSA), in accordance with MIL-STD-18.48 , will be used iteratively throughout the acquisition program as an integral part of the systems engineering process. The logistics support analysis process will be used to: develop and define supportability related design factors; and ensure the development of a fully integrated system support structure.

Manpower, personnel, training, and safety are essential design, human systems integration, and support considerations. They will be given explicit attention early in the acquisition process.

Relationship to the System Acquisition Process

DoD 5000.2R (section C5.2.3.5.4.1. Supportability Analyses.) requires that the PMs shall conduct supportability analyses as an integral part of the systems engineering process, beginning at program initiation and continuing throughout the program life cycle. The results of these analyses shall form the basis for the related design requirements included in the system performance specification and in the documentation of logistics support planning. The results shall also support subsequent decisions to achieve cost-effective support throughout the system life cycle. For products, this includes all new procurements and major modifications and upgrades, as well as reprocurement of systems, subsystems, components, spares, and services that are procured beyond the initial production contract award. PMs shall permit broad flexibility in contractor proposals to achieve program supportability objectives.

Readiness Objectives. Preliminary peacetime and wartime readiness objectives and thresholds will be established by Milestone I, Concept Demonstration Approval, and final objectives and thresholds will be established by Milestone II, Development Approval. The acquisition strategy will identify resource requirements and include explicit planning for achieving these objectives.

Inputs

Supportability requirements/constraints

Outputs

Supportaility concepts and criteria

Activities

8.5.1 Identify procedures/documentation/decision aiding design concepts

8.5.1.1 Develop Procedures/Documentation requirements- identify information presentation design requirements for information format, source, quantity, currency, and update rate.

8.5.1.2 Develop Procedures/Documentation Concepts ­ determine concepts for use of hard copy (manuals, checklists, etc.) or electronic documentation (hand held or desktop computer display)

8.5.1.3 develop tradeoff criteria from requirements

8.5.1.4 conduct tradeoffs

8.5.1.5 Develop Documentation Design Criteria - integrate selected concepts over missions, conditions and sequences, and develop design criteria for selected concepts

8.5.2 analyze requirements, concepts and criteria for the workstation design for supportability

8.5.2.1 design of technical documentation

8.5.2.2 design for handling and transportation

8.6 Identify Requirements for Human-Machine Interface Design for Habitability and Quality of Life

Overview: Habitability design involves specifying workspace free volume, environmental effects, traffic patterns, workspace layout, facility compartmentalization, and adequacy of the design for habitability. The human factors engineering concepts to be developed will address the major user-machine and user-facility interface issues. Human factors engineering concepts will either be developed or will reflect an assessment of architectural/engineering design concepts from a human factors engineering point of view. Specific concepts will include the following:

Relationship to the Systems Acquisition Process

One of the elements of ILS is facilities.

Inputs

Requirements for habitability

Outputs

Habitability concepts and criteria

SUBSTEPS/ACTIVITIES/GUIDELINES

8.6.1 review task analyses to identify where tasks must be performed in a facility

8.6.1.1 allocate tasks and task performance requirements to facilities

8.6.1.2 identify facility user functions and associated facility requirements

8.6.2 Review facility design concepts

8.6.2.1 review facility design concepts in predecessor/baseline systems

8.6.2.2 review facility design problems from fleet feedback system

8.6.2.3 conduct walkthrough of task sequences

8.6.3 develop alternative facility concepts

8.6.4 conduct tradeoffs

8.6.5 integrate facility concept selection for different task sequences

8.6.6 Identify quality of life design requirements

5

Step 9 Identify Initial Manning Estimates and Develop Manning/Skill Reduction Concepts

Overview: The Manpower Estimate shall: outline the official manpower position; address whether the program is affordable from a military end strength and civilian work year perspective; clearly state the risks associated with achieving manpower numbers reported in the estimate; and consider the program objectives, but shall base the estimate on a careful assessment of the risks and a realistic appraisal of the level of improvements most likely to be realized.

The manpower estimate shall report the total number of personnel needed to operate, maintain, support, and provide training for the program upon full operational deployment. It shall report the number of military (officer, warrant officer, and enlisted), DoD civilian, and contract manpower requirements for each fiscal year of the program beginning with initial fielding and ending with full operational deployment. A separate estimate should be provided for each Component (for joint programs) and separately for the Active, Reserve, and National Guard forces.

The estimate shall report manpower requirements and authorizations (as military end-strengths and civilian work years) for each fiscal year, and shall indicate if there are any resource shortfalls for any fiscal year covered in the report. The report shall state whether any increase in military end strengths or civilian work years (beyond what is included in the Future Years Defense Program) or whether waivers to existing manpower constraints will be required to support full operational deployment of the system. The report shall also address whether the manpower requirements represent an increase over what was required for the predecessor (replaced) system(s), as appropriate, and whether the manpower objectives and thresholds in the ORD, if established, were met or exceeded. For ACAT ID programs, the office of the Under Secretary of Defense for Personnel and Readiness shall review the report and provide an assessment to the OIPT.

The systems acquisition policies and procedures described in DoD 5000.1 and DoD 5000.2 place heavy emphasis on system affordability, i.e. a new system, an NDI, or a product improvement must have demonstrated affordability. HSI plays a major role in achieving system affordability by virtue of the fact that approximately 60% of a system's life cycle costs are associated with manning and training of system personnel. System's affordability will therefore be enhanced to the extent that manning and skill levels can be reduced.

The approach to defining feasible manning/skill reduction concepts essentially begins with a determination of the optimum role of the human in the system. This approach addresses the issue of establishing the optimum role of the human in a three step process: 1) identifying candidate roles of the human; 2) identifying specific requirements attendant to these roles; and 3) modeling human performance as expected in the selected set of assigned roles. In dealing with human-computer systems it is important to realize that the issue is not so much defining the allocation of system functions or tasks to human or machine performance as establishing the role of human in the system. In a human-machine system where both components are equally competent to perform individual functions and tasks, the design issue is to determine the role of the human vs automation in the performance of each function or task. The emphasis on the role of human in the system acknowledges the fact that the human has some role in every system function or task. In some cases that role may encompass actual performance of the function or task. It is also important to realize that an assigned role for human performance may change with changes in operational conditions. Thus a task optimally performed by a human under certain conditions of workload, time constraints, or task priority, may be more optimally automated under other conditions. It is also important to keep in mind that automating a function or task does not logically mean that the human does not have a role, that he or she has effectively been designed out of the system for that specific function of task. Rather, in an automated function or task, the role of the human is that of a manager, monitor, decision maker, system integrator, or backup performer.

Relationship to the System Acquisition Process

Operational constraints will be considered in the evaluation of alternative concepts. For those constraints relevant to the preferred concept(s), an initial list of critical system characteristics with proposed thresholds and objectives will be identified in the Operational Requirements Document. Selected parameters will be included in the Concept Baseline and the Test and Evaluation Master Plan. Critical System Characteristics include manning characteristics which will be identified to account for the numbers and skills of available people considering operational safety, security, and manpower restrictions.

Inputs

Lessons learned from existing systems

Results of function allocations and role of human determinations

Outputs

Manning reduction concepts with estimates of the magnitude of the reduction

SUBSTEPS/ACTIVITIES/GUIDELINES

9.1 Identify the Initial manning Estimate

9.1.1 Assemble task sequences in the BCS

9.1.2 Identify the roles of the human vs automation in the BCS

9.1.3 Identify the roles of the human vs automation in the emerging system

9.1.4 Define the network of tasks in the emerging system

9.1.5 Establish manning estimates by subsystem and condition of readiness

9.1.6 Establish manning estimates for functions and tasks which cross subsystems or which are at the total ship level

9.1.7 Compile manning estimates into the Initial manning Estimate

9.2 Investigate alternate manpower/skill reduction options

9.2.1 Identify the optimum roles of the human in the system as in Step 1.5

9.2.2 Establish the task network associated with these roles as in Step 1.8

9.2.3 Establish the workloads and manning levels associated with these roles as in Step 1.9

9.2.4 Identify task areas for which reduced manning is feasible

9.2.5 Determine the impact of cross training on manpower/skill reduction

9.2.6 Determine the impact of automation of manual operations on manpower/skill reduction

9.2.7 Determine the impact of improved design on manpower/skill reduction

9.2.8 Determine the impact of changes in policy and doctrine on manpower/skill reduction

9.3 Determine the potential for improved design to reduce workload and manning.

9.3.1 Determine the potential for display design

9.3.1.1 Determine requirements for displays which are meaningful, readable, integrated, accurate, current, complete, clear, directive, transparent, readily associated with control actions and other related displays, and responsive to information requirements;

9.3.1.2 Identify problems with display design in the BCS

9.3.1.3 Identify display design improvements

9.3.1.4 Identify how display improvements will reduce workload and/or reduce the likelihood of human error

9.3.2 Determine the potential for control design

9.3.2.1 Determine requirements for controls which are reachable, identifiable, operable, consistent, compatible with expectations and conventions, and simple to use;

9.3.2.2 Identify problems with control design in the BCS

9.3.2.3 Identify control design improvements

9.3.2.4 Identify how control improvements will reduce workload and/or reduce the likelihood of human error

9.3.3 Determine the potential for console and panel design

9.3.3.1 Determine requirements for consoles and panels including required control and display functions arranged in terms of functions, sequence of operations, and priorities;

9.3.3.2 Identify problems with console and panel design in the BCS

9.3.3.3 Identify console and panel design improvements

9.3.3.4 Identify how console and panel design improvements will reduce workload and/or reduce the likelihood of human error

9.3.4 Determine the potential for procedures design

9.3.4.1 Determine requirements for procedures which are logical, consistent, straightforward, and which provide feedback;

9.3.4.2 Identify problems with procedures design in the BCS

9.3.4.3 Identify procedures design improvements

9.3.4.4 Identify how procedures improvements will reduce workload and/or reduce the likelihood of human error

9.3.5 Determine the potential for communications design

9.3.5.1 Determine requirements for communications which are standardized, consistent, intelligible, clear, concise, identifiable, prioritized, and available;

9.3.5.2 Identify problems with communications design in the BCS

9.3.5.3 Identify communications design improvements

9.3.5.4 Identify how communications improvements will reduce workload and/or reduce the likelihood of human error

9.3.6 Determine the potential for environmental design

9.3.6.1 Determine requirements for environments which are within performance, comfort and safety limits, designed in terms of task requirements, and consider long term as well as short term exposure.

9.3.6.2 Identify problems with environmental design in the BCS

9.3.6.3 Identify environmental design improvements

9.3.6.4 Identify how environmental design improvements will reduce workload and/or reduce the likelihood of human error

9.4 Determine the potential for task simplification to reduce workload/manning.

9.4.1 Identify the potential for reducIng task demands (physical, cognitive, and perceptual-motor) by reducing the:

9.4.4.1 amount of information to be processed,

9.4.4.2 complexity of the information processing,

9.4.4.3 number of decisions and options to be handled,

9.4.4.4 complexity of actions,

9.4.4.5 needs for interactions with other operators,

9.4.4.6 extent and complexity of communications,

9.4.4.7 task performance accuracy required,

9.4.4.8 special skills and knowledge required,

9.4.4.9 levels of skills such as reading comprehension,

9.4.4.10 level of stress associated with the performance of tasks under representative mission conditions,

9.4.4.11 Reduce the time constraints.

9.4.2 Identify how task simplification will reduce workload and/or reduce the likelihood of human error

9.5 Determine the potential for automation to reduce manning

9.5.1 Determine for which tasks is increased automation feasible

9.5.2 Identify the role of the human for tasks for which the level of automation has been increased

9.5.3 Identify how automation will reduce workload and/or reduce the likelihood of human error

9.6 Determine the potential for cross training to reduce manning

9.6.1 Determine for which tasks is cross training feasible

9.6.2 Identify the role of the human for tasks for which cross training has been implemented

9.6.3 Identify how cross training will reduce workload and/or reduce the likelihood of human error

9.7 Identify safety concerns with reduced manning

9.7.1 Fatigue due to job factors

9.7.1.1 working hours - duration of the workday

9.7.1.2 watch rotation - work/rest cycles

9.7.2 Fatigue due to task overloading

9.7.3 Emergency response

9.7.3.1 all hands emergency - fire, explosion, collision, grounding

9.7.3.2 safety procedures

9.7.3.3 continue to operate safely in situations of power loss or failure of vital equipment, i.e. steering gear, navigation equipment, mooring equipment, propulsion, and cargo gear

9.7.4 Training impacts on safety

9.7.4.1 reduced training opportunities for unlicensed personnel

9.7.4.2 members of smaller crews will need to be more broadly skilled

9.7.4.3 training will need to be updated more often as technology advances

9.7.5 Service continuity by crew members

9.7.6 Physical demands, e.g. strength

9.7.7 Shipboard social conditions, e.g. little if any shore time

9.8 Determine the impact of automation on ship systems

9.8.1 Integrated bridge systems

9.8.1.1 expanded use of decision aiding - expert systems

9.8.1.2 single-human bridge - watch officer also serves as helmsman and lookout

9.8.1.3 potential problems of circadian rhythms

9.8.2 Deck automation

9.8.2.1 automated mooring winches

9.8.2.2 automated hatch cover securing

9.8.2.3 interior communications

9.8.3 Engine room automation

9.8.3.1 control from bridge or other central location

9.8.3.2 periodically unmanned control room

9.8.3.3 ship must carry sufficient personnel to take over in the event of automated systems failures

9.9 Determine impact of reduced manning on training/certification

9.9.1 Need for cross-trained personnel

9.9.2 Need for general purpose ratings - across departments

9.9.3 Need for a maintenance training option

9.10 Identify operating conditions that impact reduced manning options

9.10.1 operating procedures - maintenance approach

9.10.2 maintenance concept

9.10.3 crew/role flexibility - use of cross-training

9.10.4 crew continuity - cohesion as a crew

9.10.5 ship familiarity - same ship or class

9.10.6 regulations/union contract provisions

9.10.7 personnel selection criteria/procedures

9.10.8 job design

9.10.9 training/proficiency

9.11 conduct workload analyses of alternate manpower reduction approaches

9.11.1 Conduct task network simulation to assess workloads as in Step 1.9

9.11.2 Assess workloads under alternate manpower reduction approaches

9.12 select promising manpower/skill reduction approaches

9.12.1 Set up tradeoffs as described in Step 1.7

9.12.2 Conduct tradeoffs

9.12.3 Identify the most promising manpower/skill reduction approach

5

Step 10 Identify Concepts for the Systems Approach to Training (SAT)

Overview: Training requirements will be derived through application of the systems approach to training (SAT). The process for applying the systems approach to training (SAT) in the development training needs for reduced manning ships must be responsive to specific driving policies and requirements. Based on the policies and requirements, certain specific characteristics of the SAT can be identified. These characteristics demand that the SAT must be: performance-based; requirements-driven; iterative; integrative; standardized, formalized, and tailorable; verifiable; supportable; and directed at risk reduction. This step is also concerned with determining training requirements in a reduced manning environment. Finally, training media/device requirements are developed.

Since both training and design are focused on enabling, enhancing, supporting and maintaining required levels of human performance capability in systems, there must be a synergistic mutual interrelationship between training and design that extends from system conceptual development through detail design, deployment and operation. Training is concerned with providing the skills and knowledge necessary for effective human performance. Design, specifically human engineering design, is concerned with providing human-machine interfaces that enable effective human performance. The synergism between training and design is manifested in a number of ways, including the following.

Training requirements should be developed concurrently with design requirements.

Requirements for human performance drive development of training programs and design concepts. In order to ensure that human performance requirements and objectives influence design, efforts to provide effective human performance, involving both training development and design, must be initiated early in system development. Throughout the system development process, the identification and analysis of training requirements must parallel the development of design concepts since they are both directed at the same objective, that of prodHCIng effective, responsive and proficient human performance.

Training and design development must proceed from the same front-end analysis.

Training development and human engineering design both depend on a front-end analysis process that begins with the analysis of missions and associated functions, and proceeds through determination of the roles and responsibilities of humans, to the analysis of requirements for performance of human assigned tasks. In the training realm, as dictated by the instructional systems development (ISD) methodology, this front-end process results in training objectives, which establish the capabilities which the human must possess at the conclusion of training. In human engineering design the front-end analysis process defines the requirements associated with the human-machine interfaces required for the exercise of the required capability. For purposes of maximizing the effectiveness, efficiency, and economy of the requirements analysis process, the same front-end analysis should serve as the foundation for training development and design decisions.

The amount of training required (the training burden) to achieve required performance capability is a direct function of the adequacy of the design.

Training is often implemented to compensate for faulty design.

As a corollary to issue above, it is also a fact that when designers fail to apply human engineering principles, methods and data in their designs, they invariably look to training alone to shoulder the effort of ensuring that humans perform at the required level of proficiency. Where this approach usually fails is in situations of tight time constraints, high stress, reduced situation awareness, or uncertainty where humans tend to forget what they have learned or are likely to be confused while operating a burdensome interface which was not designed in terms of their requirements, capabilities and limitations.

The extent and manner in which training media should faithfully reflect the current design varies as a function of the training objective.

The issue of training fidelity, or the concordance between the training situation and the actual system, reflects another close association between training and design. Where the training objective is to provide expertise in following procedures, the degree of physical fidelity, the agreement between the physical human-machine interfaces in the training situation and in the system, must be high. However, if the training objective is to provide expertise on cognitive capabilities, such as decision making, diagnosis, maintaining situation awareness, and integration of information from numerous sources, it is conceptual fidelity rather than physical fidelity that must be maximized. Conceptual fidelity refers to the fidelity of the training system's representation of the battlespace situation, and the response of the warfare system in that battlespace

Training effectiveness and design effectiveness decisions should be based on the same metrics.

The measurement of training effectiveness, the extent to which training objectives are met and training is acquired and retained, constitutes first and foremost the measurement of human performance capability at a point in time, and over time. The measurement of the effectiveness of human engineering design, conducted in test and evaluation exercises, also reflects the scaling of human performance with respect to a criterion. These metrics should be the same, since the underlying system attribute being measured, human performance, is the same.

Embedded, organic training provisions must be designed into the system.

Embedded training represents a situation wherein both physical fidelity and conceptual fidelity is maximized by virtue of the fact that the interface used for training is the system interface, and the training environment is the actual system operating environment. Provisions for embedded training must be addressed from the earliest stages of system development, in the same process in which human-machine interfaces are designed into the system.

System design should include provisions for refresher training to restore perishable skills.

Training requirements must address the need for refreshing perishable skills, those skills which are rarely exercised, such as those associated with conducting maintenance on highly reliable equipment of conducting infrequently required operations. In order to provide refreshment training, provisions for such training must be integrated into the system design.

System design should include provisions for team training such as crew resource management.

The conduct of operations requiring team performance, and communication, and collaboration among team members, will require human interfaces and skills training different than that directed at individual human performance. Design of the human interfaces will require attention to facilitating effective team training and crew resource management training throughout the life cycle of the system.

Both training and design are concerned with providing knowledge.

Another way in which training and design are closely related is in the generation of knowledge. In existing systems a system development tradeoff addressed the issue of placing knowledge in the head (through training) or in the book (job performance aids). Through the use of job performance aiding, some knowledge was only provided to the user when it was needed, such as how to disassemble a complex pump. In more advanced systems, knowledge will be provided to the human through decision support systems, operator's associates, and system displays. The system will need to be designed to generate and present this knowledge, and the integration of such just-in-time knowledge with knowledge acquired through training will become an important design issue. The concept of intelligent tutoring is already being designed into complex systems to provide knowledge that is tailored to the individual user and his or her specific need.

Relationship to the System Acquisition Process: This step outputs information to be used in the development of the Navy Training System Plan.

Inputs

Training requirements and alternative training system concepts to be evaluated.

Outputs

Results of training requirements analysis, with emphasis on the impact of reduced manning, training device requirements, and training system evaluation.

SUBSTEPS/ACTIVITIES/GUIDELINES

10.1 Develop Manning and Skill Estimates

10.1.1 review task analysis data

10.1.2 identify the LSAR manning levels by tasks

10.1.3 identify skill requirements by tasks

10.1.4 integrate manning and skill requirements across task sequences

10.2 Apply the Systems Approach to Training to identify training requirements.