## Info

Prepare Monte Carlo output plots and data

The next step is to run more simulations using new start times or otherwise varying the conditions for the scenarios. Changing the start times alters the relative timing and geometry between the satellites and the events they are observing, thus, averaging results caused by these characteristics. Collecting statistics on multiple runs is called a Monte Carlo simulation. For example, we might average the percentage of forest fires detected over different runs with different timing, but on the same scenario, to estimate the overall probability of detecting forest fires—our ultimate measure of effectiveness. The system simulator should accumulate output statistics and prepare output plots over the Monte Carlo runs.

Frequently, in running mission simulations, we must choose between realistic and analytical scenarios. Realistic scenarios usually are too complex to help us understand how the system works but are still necessary to satisfy the end users. On the other hand, simple scenarios illuminate how the system is working but do not show how it will work in a real situation. The best answer is to use simple scenarios for analysis and realistic scenarios to assess mission performance. In the FireSat example, we might begin by studying a single satellite to determine how it behaves and then expand to a more complex simulation with several satellites. We might also start evaluating a multi-satellite constellation by looking at its response to a simple situation, such as one fire or a small group of uniformly distributed fires. This trial run will suggest how the system performs and how changes affect it. We can then apply this understanding as we develop more realistic simulations.

A related problem concerns using a baseline scenario to compare options and designs. Repeating a single scenario allows us to understand the scenario and the system's response to it We can also establish quantitative differences by showing how different designs respond to the same scenario. But this approach tends to mask characteristics that might arise solely from a particular scenario. Thus, we must understand what happens as the baseline changes and watch for chance results developing from our choice of a particular baseline scenario.

Finally, mission simulations must generate usable and understandable information for decision makers—information that provides physical insight Two examples are strip charts of various system characteristics and animated output A strip chart plot is similar to the output of a seismograph or any multi-pin plotter, in which various characteristics are plotted as a function of time. These characteristics might include, for example, whether a particular satellite is in eclipse, how much time it spends in active observation, and the spacecraft attitude during a particular time step. Plots of this type give a good feel for the flow of events as the simulation proceeds.

A valuable alternative for understanding the flow of events is looking at an animation of the output, such as a picture of the Earth showing various changes in the target, background, and observation geometry as the satellites fly overhead. Thus, as Fig. 3-4 illustrates, an animated simulation of FireSat output could be a map of a fire-sensitive region with areas changing color as fires begin, lines showing satellite coverage, and indications as to when fires are first detected or when mapping of fires occurs. Animation is not as numerical as statistical data, but it shows more clearly how the satellite system is working and how well it will meet broad objectives. Thus, mission analysts and end users can assess the system's performance, strengths and shortcomings, and the changes needed to make it work better.

333 Commercial Mission Analysis and Mission Utility Tools

' Creating a mission utility simulation for your specific mission or mission concept is both time consuming and expensive. It is not uncommon for the simulation to be completed at nearly the same time as the end of the study, such that there is relatively little time to use the simulation to effectively explore the multitude of options available to the innovative system designer.

In my view, the single largest step in reducing software cost and risk is the use of commercial, off-the-shelf (COTS) software. The basic role of COTS software in space is to spread the development cost over multiple programs and reduce the risk by using software that has been tested and used many times before. Because the number of purchasers of space software is extremely small, the savings will be nowhere near as large as for commercial word processors. Nonetheless, reductions in cost, schedule, and risk can be substantial. Most COTS software should be at least 5 times cheaper than program-unique software and is typically 10 or more times less expensive. In addition, COTS software will ordinarily have much better documentation and user interfaces and will be more flexible and robust, able to support various missions and circumstances.

The use of COTS software is growing, but most large companies and government agencies still develop their own space-related software for several reasons. One of the

Fig. 3-4. Hypothetical Animation Output for FIreSat Mission Utility Simulator. Color displays are very valuable for animation sequences because we need to convey multiple parameters hi each frame.

best ways to develop and maintain expertise is to create your own systems and models. Thus, organizations may want to support their own software group, particularly when money is tight Also, it's hard to overcome the perception that it costs less to incrementally upgrade one's own system than to bear the cost and uncertainty of new COTS tools. In this trade, die "home built" systems often don't include maintenance costs. Finally, customers often don't know which COTS tools aie available. Professional aerospace software doesn't appear in normal software outlets, advertising budgets are small, and most information is word-of-mouth through people already in the community. Despite these substantial obstacles, many organizations are now using COTS software in response to the strong demand to reduce cost

In order to use COTS tools to reduce space system cost, we need to change the way we use software. We need to adapt to software not being exactly what we want, look for ways to make existing software satisfy the need, or modify COTS software to more closely match requirements. This is a normal part of doing business in other fields. Very few firms choose to write their own word processor, even though no single word processor precisely meets all needs. Instead, they choose one that most closely matches what they want in terms of functions, support, and ease of use. We should use the same criteria for COTS space software. In addition, we need to set realistic expectations concerning what COTS software can do. Clearly, we can't expect the low prices and extensive support that buyers of globally marketed commercial software enjoy. We have to adjust our expectations to the smaller market for space-related software, which means costs will be much higher than for normal commercial products. Maintenance and upgrades will ordinarily require an ongoing maintenance contract Within the aerospace community, a standard arrangement is for a maintenance and upgrade contract to cost 15% of the purchase price per year.

Using COTS software and reusing existing noncommercial software requires a different mindset than continuously redeveloping software. We need to understand both the strengths and weaknesses of the relatively small space commercial software industry. Because the number of copies sold is small, most space software companies are cottage industries with a small staff and limited resources. We shouldn't expect space-software developers to change their products at no cost to meet unique needs. For example, it would be unrealistic to expect a vendor of commercial software for low-Earth orbit spacecraft to modify the software for interplanetary missions at no cost, because few groups will buy interplanetary software. On the other hand, the small size of the industry means developers are eager to satisfy the customers' needs, so most are willing to work with their customer and to accept contracts to modify their products for specific applications. This can be far less expensive than developing software completely from scratch.

There is a hierarchy of software cost, going from using COTS software as is, to developing an entirely new system. In order of increasing cost, the main options are

1. Use COTS software as sold

2. Use COTS software libraries

3. Modify COTS software to meet specific program needs (modification may be done by mission developer, prime contractor, or software developer)

4. Reuse existing flight or ground software systems or modules

5. Develop new systems based largely on existing software components

6. Develop new systems from scratch using formal requirements and development processes

This hierarchy contains several potential traps. It may seem that the most economical approach would be for the prime contractor or end-user to modify COTS software to meet their needs. However, it is likely that the COTS software developer is in a better position to make modifications economically and quickly. Although the end-users are more familiar with the objectives and the mission, the software developer is more familiar with the organization and structure of the existing code.

Secondly, there is frequently a strong desire to reuse existing code. This will likely be cheaper if the code was developed to be maintainable and the developers are still available. On the other hand, for project-unique code developed with no requirement for maintainability, it may be cheaper, more efficient, and less risky simply to discard the old software and begin again.

Commercial mission analysis tools fall into three broad categories, each of which is described below. Representative examples of these tools are listed in Table 3-9.

Generic Analysis Systems. These are programs, such as MalLab™, which are intended to allow analysis and simulation of a wide variety of engineering and science problems. They typically cost a few hundred to several thousand dollars and can dramatically reduce the time needed to create simulations and analyze the results. Because these are generic tools, specific simulation characteristics are set up by the user, although subroutine libraries often exist Thus, we will need to create orbit propagators, attitude models, environment models, and whatever else the problem dictates. We use this type of simulation principally for obtaining mathematical data and typically not for animation.

Law-Cost Analysis Programs. These are programs intended for a much wider audience such as the amateur astronomy or space science community. However, when carefully selected and used appropriately, they can provide nearly instant results at very low cost The programs themselves cost a few hundred dollars or less, are

TABLE 3-9. Commercial Space Mission Analysis and Design Software. New versions are typically released roughly annually. Because of the very small size of the space market, commercial space software both enters and leaves the marketplace on a regular basis.

Product

Publisher

Approx. Cost

Purpose

Dance of the Planets

Arc Science Simulations

\$250

Amateur visual and gravitational model of the solar system useful for interplanetary mission design

Edge

Autometric

\$5,000 +

Professional mission analysis system; many modules; can be customized

0 0