Search This Blog

Friday, July 8, 2011

Methods for Verifying System Performance

9.2 Verifying that System Requirements are Met
The first phase of verifying system requirements is a formal engineering process that starts with requirements analysis and ends when the system is accepted by its customer.  During the system integration and testing steps must be taken to verify that the system satisfies every “shall” statement in the requirements. These shall statement requirements are collected in a document called the Verification Matrix. The results of the integration and testing of these requirements are documented in a Compliance Matrix. The integration and testing is defined and planned in a System Integration and Test Plan. Related documentation includes the Test Architecture Definition, hardware and software Test Plans & Procedures and Test Data Analysis Plans.
The roles of systems engineers in verification include:
  • Developing the optimum test strategy and methodology and incorporating it into the design as it is developed
  • Developing the top level System Integration and Test Plan
  • Developing the hierarchy of Integration and Test Plans from component level to system level
  • Documenting all key system and subsystem level tests
  • Defining system and subsystem level test equipment needed and developing test architecture designs
  • Developing the Test Data Analysis Plans
  • Analyzing test data
  • Ensuring that all shall requirements are verified and documented in the Compliance Matrix.
Good systems engineering practice requires that requirements verification takes place in parallel with requirements definition. The decision to define a requirement with a “shall” or “may” or “should” statement involves deciding if the requirement must be verified and if so how the requirement will be verified. This means that the requirements verification matrix should be developed in parallel with the system requirements documentation and reviewed when the system requirements are reviewed, e.g. at peer reviews and at a formal System Requirements Review (SRR).
9.2.1 Verification Matrix – The verification matrix is documentation that defines for each requirement the verification method, the level and type of unit for which the verification is to be performed and any special conditions for the verification. Modern requirements management tools facilitate developing the verification matrix. If such tools are not used then the verification matrix can be developed using standard spreadsheet tools.
There are standard verification methods used by systems engineers. These methods are:
  1. Analysis -Verifies conformance to required performance by the use of analysis based on verified analytical tools, modeling or simulations that predict the performance of the design with calculated data or data from lower level component or subsystem testing. Used when physical hardware and/or software is not available or not cost effective.
  2. Inspection - Visually verifies form, fit and configuration of the hardware and of software. Often involves measurement tools for measuring dimensions, mass and physical characteristics.
  3. Demonstration - Verifies the required operability of hardware and software without the aid of test devices. If test devices should be required they are selected so as to not contribute to the results of the demonstration.
  4. Test - Verifies conformance to required performance, physical characteristics and design construction features by techniques using test equipment or test devices. Intended to be a detailed quantification of performance.
  5. Similarity - Verifies requirement satisfaction based on certified usage of similar components under identical or harsher operating conditions.
  6. Design – Used when compliance is obvious from the design, e.g. “The system shall have two modes, standby and operation”.
  7. Simulation – Compliance applies to a finished data product after calibration or processing with system algorithms. May be only way to demonstrate compliance.
The DoD SEF defines only the first four of the methods listed above. Many experienced systems engineers find these four too restrictive and also use the other three methods listed. To illustrate a verification matrix with an example consider the function Switch Power. This function might be decomposed as shown in Figure 9-1.

Figure 9-1 A function Switch Power might be decomposed into four sub functions.
An example verification matrix for the functions shown in Figure 9-1 is shown in Figure 9-2. In this example the switch power function is assumed to be implemented in a switch module and that both an engineering model and a manufacturing prototype are constructed and tested. In this example no verification of the switch module itself is specified for production models. Verification of the module performance for production modules is assumed to be included in other system level tests.
It’s not important whether the verification matrix is generated automatically from requirements management software or by copy and paste from a requirements spreadsheet. What is important is to not to have to reenter requirements from the requirements document to the verification matrix as this opens the door for simple typing mistakes.

Figure 9-2 An example verification matrix for a switch module.


  1. Interesting article. It is quite unfortunate that over the last several years, the travel
    industry has had to take on terrorism, SARS, tsunamis, bird flu virus, swine flu, and the first
    ever real global downturn. Through everthing the industry has really proven to be sturdy,
    resilient and dynamic, getting new solutions to deal with hardship

    Also visit my web page -

  2. I have noticed that car insurance companies know the automobiles which are at risk of accidents and various risks.
    In addition, they know what kind of cars are susceptible to higher risk along with the higher risk they have the higher
    the premium rate. Understanding the easy basics regarding car insurance will let you choose the right form of insurance policy that could take care of your requirements in case you
    become involved in any accident. Thank you sharing the ideas on your blog.

    My webpage ... 100 free dating site

  3. Hey very cool blog!! Man .. Excellent .. Amazing .

    . I'll bookmark your web site and take the feeds also�I am happy to find numerous useful info here in the post, we need develop more techniques in this regard, thanks for sharing. . . . . .

    Also visit my web blog :: singapore dating

  4. Good post. I study one thing tougher on different blogs everyday.
    It'll always be stimulating to learn content from other writers and observe a bit of something from their store. I�d choose to use some with the content on my blog whether or not you don�t mind. Natually I�ll provide you with a link on your web blog. Thanks for sharing.

    Also visit my site :: best dateing site

  5. Spot on with this write-up, I really think this web site needs way more consideration.
    I�ll probably be again to read rather more, thanks for that info.

    Here is my web blog;

  6. people who are open to high risk regarding developing long term asbestos connected malignancies.
    Thanks for revealing your ideas for this important health issue.

    Feel free to surf to my web-site adult finder

  7. Superb website you have here but I was curious about if you knew of any message
    boards that cover the same topics talked about here?

    I'd really like to be a part of group where I can get feedback from other experienced people that share the same interest. If you have any suggestions, please let me know. Kudos!

    my web page; skype sex

  8. Sweet blog! I found it while searching on Yahoo News.
    Do you have any tips on how to get listed in Yahoo News? I've been trying for a while but I never seem to get there! Thank you

    My web site - free adult dating sites

  9. hello!,I like your writing so much! share
    we communicate more about your post on AOL? I require an expert on this
    area to solve my problem. Maybe that's you! Looking forward to see you.

    Review my website - sex.fisbok

  10. I have observed that in the world the present moment, video games are definitely the latest popularity with children of all ages.
    Often times it may be out of the question to drag the kids away from the games.
    If you want the best of both worlds, there are several educational games for kids.
    Thanks for your post.

    Here is my blog ... singles dating sites for free

  11. Its such as you read my thoughts! You appear to know so much about this, such
    as you wrote the guide in it or something. I believe that you
    just can do with some p.c. to pressure the message home a little bit, however instead of that, this
    is fantastic blog. An excellent read. I'll certainly be back.

    Here is my web site; Suresh Gaddam