PEP

[¤9] Version	[¤10] Date	[¤11] Author	[¤12] Change Description
[¤13] 0.5	[¤14] 02 July 08	[¤15] Karl Schopmeyer	[¤16] Initial Submission
[¤17]	[¤18]	[¤19]	[¤20]

[¤24] The Pegasus CVS includes both the source for Pegasus (along with its make definitions) and the source and expected test results for a suite of unit and end-end tests of Pegasus. These tests are intended to be used as part of the Pegasus development process to insure that a) functionality works as plans, b) code changes to to not regress the status of existing functionality.

[¤25] In addition to the test programs, data, etc. the OpenPegasus project provides a web site where regular results from testing during development can be recorded so that there is visible proof that the tests work and a basis for working on issues that might arise that cause test failures.

[¤26] Description of the Pegasus Test Environment

[¤27] Unit Tests - Tests run against particular components of Pegasus software that test that component's functionality. Typically these tests are defined in a subdirectory of the software with the name test. Thus, for example the pegasus/src/Pegasus/Common functions have an extensive set of tests in subdirectories of pegasus/src.Pegasus/Common/tests. Unit test should NOT require a running Pegasus server, an existing repository and should require a minimum of other Pegasus components that the tested component.

[¤28] End-to-End Tests - These are tests of the running server. The are located in a number of places throughout the source tree including

[¤32] These tests should be able to depend on the server running and a repository in existence. Note, however, that except for those tests in the pegasus/tests directory, they should not depend on a particular version of the CIM Schema in the repository.

[¤33] Extended Test Suites - These tests are part of the TestMakeFile that require that particular configuration options be set for the server. They may stop the server, set options and restart the server either to run special tests or possibly to rerun the whole end-to-end set of tests.

[¤35] The Nightly Build Reporting Environment

[¤36] The Nightly build reporting environment is a web page that reports nightly test status received via email defining the status of nightly tests run on selected platforms. See the link http://cvs.opengroup.org/cgi-bin/pegasus-build-status.cgi to view the current nightly build status page. Normally members of the projects run the Pegasus test suite each night and use a specifically formatted email to report the results by sending a structure email with the test results to the opengroup. The OpenGroup provides software that analyzes the body of messages address to a particular address and creates entries in for the Pegasus Build Status web page.. The following information is reported on this web page for each email received.

[¤43] Nightly Build Email Format

[¤44] To create an entry (or update an existing entry) in the nightly build status table, a testor runs the test suite and creates an email to define a) specific fields defining the status that will go into the table, b) additional information from the test which provides background on the tests, errors, etc.

[¤45] This email MUST contain specifically formatted fields to define the information that will be placed into the web page as follows:

[¤46] Subject Line

[¤47] This line defines the completion status <SUCCESS | FAILURE, the platform and the Date of the test in the format yymmdd. The subject line is not used by the software that analyzes the messages for the Build Status web page but can be useful as a summary if the email is sent to any other destinations.

[¤48] Message Text

[¤49] The beginning of the message MUST contain the following name value pairs in the form NAME ":" value

[¤50] Platform definition

[¤51] This defines the platform for this test. It can be noted from the existing web page that there are a variety of information types in this field. However, it should clearly identify the platform base for the test including the platform definition used by Pegasus, and any manufacturer name/ version information that will make this field unique. One format used is to provide the name of the platform file (ex LINUX_IX86_GNU with further infomation about mfgr and version in parenthesis (ex. LINUX_IX86_GNU (REHEL4))

[¤52]     "Platform:" <platform definitions consisting of the platform file name with optional platform mfgr name>

[¤53] CVS Branch definition

[¤54] CVS branch used for the test. This allows the nightly build test to separate out the various development and release branches in the display. Some examples include MAIN, RELEASE_2_7-branch.

[¤55]     "Branch:" <<cvs tag used for the checkout for this test>>.

[¤56] Date Definition

[¤57] Date of the test.

[¤58]     "Date:" <<date of the test in the form ddmmyy>>

[¤59] Time Definition

[¤60] Time at which the test was started.

[¤61]     "Time:" <hhmm> <<time zone>

[¤62] Note that there is no definition today of the timezone parameter

[¤63] Status Definition

[¤64] This is a keyword field that defines whether the test was a success or failure. The expected way these tests are run is to simply run until the first error and terminate the test. If the checkout, configure, build, and test is successful, the test is considered successful. If there is any failure the test is considered a failure and the test should stop at that point.   Note that the build scripts in OpenPegasus (ex. make new world) do exactly that.

[¤65]     "Status:" <<"SUCCESS" | "FAILURE">>

[¤66] Options Definition

[¤67] This defines the build options that were used for the test in simple one line keyword format. Today it is not displayed on the build status page but this is planned for the future. Note that at this time, the exact form of the keywords for the options has NOT been defined.

[¤68]     "Options: <comma separate list of the build options>>

[¤69] Contributor Definition

[¤70] Name of the contributor. This is an identifier of the organization that submitted this test result set.

[¤71]     "Contributor:" <<name>>

[¤72] The exact form and name list has not been settled at this point

[¤73] Examples of the Test Results Email

[¤74] Subject: SUCCESS: HEAD: LINUX_X86_64_GNU (FEDORA 6): 080710

[¤75] Platform: LINUX_X86_64_GNU (FEDORA 6)

Branch: MAIN

Date: 080710

Time: 0609 GMT

Status: SUCCESS

End Time: 0753 GMT

[¤76] << any more detailed information can follow here >>

[¤79] Test Results Email Transmission

[¤84] Note that in addition, it MUST have a From address that the Open Group considers a valid member address or the email will be rejected by the OpenGroup email servers upon entry. This is a limit imposed by the Open Group to eliminate unwanted messages, spam, etc.

[¤86] The complete email body will be displayable at on the Nightly Build status page. Generally the additional information that should be included in the email is that information that

[¤88] b. Help the user understand what went wrong in the case of a FAILURE status.

[¤89] Thus, for example the following information would be useful in the body of the message:

[¤93] Discussion

[¤94] <Additional information from discussions, reviews, and decisions of the steering committee and architecture team can be recorded here. Information in this section is to help reviewers but MUST also be reflected in the Proposed Solution / Rationale, etc. sections. This section is for information only during the review process.>

[¤95]
[¤96] Copyright (c) 2006 Hewlett-Packard Development Company, L.P.; IBM Corp.;
[¤97] EMC Corporation; Symantec Corporation; The Open Group.
[¤98]
[¤99] Permission is hereby granted, free of charge, to any person obtaining a copy
[¤100] of this software and associated documentation files (the "Software"), to
[¤101] deal in the Software without restriction, including without limitation the
[¤102] rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
[¤103] sell copies of the Software, and to permit persons to whom the Software is
[¤104] furnished to do so, subject to the following conditions:
[¤105]
[¤106] THE ABOVE COPYRIGHT NOTICE AND THIS PERMISSION NOTICE SHALL BE INCLUDED IN
[¤107] ALL COPIES OR SUBSTANTIAL PORTIONS OF THE SOFTWARE. THE SOFTWARE IS PROVIDED
[¤108] "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT
[¤109] LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR
[¤110] PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
[¤111] HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
[¤112] ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
[¤113] WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

[¤115] Template last modified: March 26th 2006 by Martin Kirk
[¤116] Template version: 1.11

[¤23] Overview