Pegasus Enhancement Proposal (PEP)

PEP #: 233

Title: Embedded Instance Support in Pegasus

Authors: Alex Dunfey (dunfey_alexander@emc.com)

Status:  draft 

Version History:

Abstract: This PEP proposes to extend the work done in Pegasus 2.5 to support embedded objects and implement support for embedded instances within Pegasus as well.

Definition of the Problem

As part of PEP 194, Pegasus 2.5 will be released with support for embedded objects, used primarily by lifecycle indications included in CIM Schema 2.9.  CIM Schema 2.10 contains several uses of the EmbeddedInstance qualifier, which is similar to the EmbeddedObject qualifier, but is obviously limited to instances and is more strongly typed by specifying the class to which an embedded instance belongs.  If the qualifier is present in a class definition, it must specify a class, and that class must exist in the same namespace as the containing class definition.  Currently, the CIM Specification does not allow a NULL value for the EmbeddedInstance qualifier to be specified in a class definition, although the absence of the EmbeddedInstance qualifier defaults to a NULL value for that qualifier (meaning that the property does not contain an EmbeddedInstance).  The members of SNIA plan to make extensive use of the additional error reporting capabilities included in CIM 2.10 through CIM_InstMethodCall indications and the CIM_Job::GetError() method, both of which transmit detailed error information via an embedded CIM_Error instance.  In addition, SNIA is working with the DMTF to add new classes and methods in CIM Schema 2.10 and 2.11 that utilize embedded instances and will be incorporating those elements into SMI-S.

Examples of the use of embedded instances in CIM Schema 2.10:

// ==================================================================
//  CIM_ExportedFileShareCapabilities
// ==================================================================
   [Experimental, Version ( "2.8.1000" ), Description (
       "ExportedFileShareCapabilities specifies the combination of "
       "properties that a FileExportService can support when creating "
       "or maintaining FileShares. Each supported combination of "
       "properties is specified by a ExportedFileShareSetting that is "
       "associated with the ExportedFileShareCapabilities element via "
       "SettingAssociatedToCapabilities. \n"
       "An ExportedFileShareCapabilities element specifies the "
       "properties supported when using it. A CreateGoal method is "
       "provided that creates an ExportedFileShareSetting element to "
       "be used as a goal when creating or modifying a file share.")]
class CIM_ExportedFileShareCapabilities : CIM_Capabilities {
...
      [Description (
          "Start a job to create an ExportedFileShareSetting from an "
          "ExportedFileShareSetting provided by the caller. If the "
          "operation completes successfully and did not require a "
          "long-running ConcreteJob, it will return 0. If 4096/0x1000 "
          "is returned, a ConcreteJob will be started to create the "
          "element. A Reference to the ConcreteJob will be returned in "
          "the output parameter Job. \n"
          "This method MAY return a CIM_Error representing that a "
          "single named property of a setting (or other) parameter "
          "(either reference or embedded object) has an invalid value "
          "or that an invalid combination of named properties of a "
          "setting (or other) parameter (either reference or embedded "
          "object) has been requested. \n"
          "If the input TemplateGoal is NULL or the empty string, this "
          "method returns a default ExportedFileShareSetting that is "
          "supported by this ExportedFileShareCapabilities. \n"
          "The output is returned as the SupportedGoal parameter. Both "
          "TemplateGoal and SupportedGoal are embedded objects and do "
          "not exist in the provider but are maintained by the client. "
          "\nIf the TemplateGoal specifies values that cannot be "
          "supported, this method must return an appropriate error and "
          "should return a best match for a SupportedGoal."),
       ValueMap { "0", "1", "2", "3", "4", "5", "6", "..", "4096",
          "4097..32767", "32768..65535" },
       Values { "Job Completed with No Error", "Not Supported",
          "Unknown", "Timeout", "Failed", "Invalid Parameter",
          "Template Goal cannot be matched.", "DMTF Reserved",
          "Method Parameters Checked - Job Started", "Method Reserved",
          "Vendor Specific" }]
   uint16 CreateGoal(
         [IN ( false ), OUT, Description (
             "Reference to the job (may be null if job completed).")]
      CIM_ConcreteJob REF Job,
         [IN, Description (
             "This is an element of the CIM_ExportedFileShareSetting "
             "class, or a derived class, encoded as a string-valued "
             "embedded object parameter. It specifies the goal element "
             "to be used for matching."),
          EmbeddedInstance ( "CIM_ExportedFileShareSetting" )]
      string TemplateGoal,
         [IN ( false ), OUT, Description (
             "This is an element of the CIM_ExportedFileShareSetting "
             "class, or a derived class, encoded as a string-valued "
             "embedded object parameter. It specifies the element that "
             "is returned as the best supported match to the "
             "TemplateGoal."),
          EmbeddedInstance ( "CIM_ExportedFileShareSetting" )]
      string SupportedGoal);
};

This proposal aims to integrate embedded instances into the Pegasus infrastructure, handling the encoding, parsing, and manipulation of embedded instance properties and parameters in the same manner as numerics, strings,  embedded objects, etc.

As with the EmbeddedObject qualifier, the EmbeddedInstance qualifier applies to Property, Parameter, and Method elements within schema class definitions, but will not be present in instances unless the SNIA interoperability flag is set.  There is no attribute in CIM-XML that identifies the class of the embedded instance: this information must be retrieved from the class definition.  Embedded instances are identifiable in a CIM-XML encoded instance by the presence of the EMBEDDEDOBJECT attribute with a value of "instance".  In addition, embedded instance properties, just like embedded object properties, cannot be keys and could be either transient or persistent.  In the examples above, the Error output parameter in the GetError() method and the TemplateGoal input parameter in the CreateGoal() method would be transient, but the SupportedGoal output parameter in the CreateGoal() method may or may not be persistent, depending on the implementation.

Proposed Solution

The solution being proposed here is mostly an extension to the solution implemented for PEP 194.  As such, the work done to add support for embedded objects can be leveraged to add support for embedded instances.  All of the locations in the source where changes were needed for embedded object support are essentially the same locations where changes need to be made for embedded instances.  Given the strong typing of EmbeddedInstance properties and parameters, however, some additional validation is needed for embedded instances that is not required for embedded objects.  The major areas of enhancement are:

1. A new type will be added to CIMType.h for embedded instances (CIMTYPE_INSTANCE) in order to differentiate them from embedded objects.  Union.h will also be modified to add CIMInstance and Array<CIMInstance> pointers to the values union.

2. The CIMValue class will be modified, allowing it to encapsulate CIMInstance objects. Specifically, constructor, get(), and set() methods will be added that accept the CIMInstance and Array<CIMInstance> types, allowing Pegasus to handle embedded instance values and array values containing embedded instances.  The semantics for the CIMInstance value type will be the same as the semantics for the CIMObject value type, including the handling of NULL values.

3. XML encoding and decoding will be modified to support embedded instances. This requires changes throughout XMLReader and XMLWriter, and also in the CIMParamValueRep::toXml() and CIMPropertyRep::toXml() methods.

4. Recognize embedded instance values in CQL filtering.  The CQLValueRep and CQLSelectStatementRep classes need to be enhanced to check for the new type CIMTYPE_INSTANCE so it can apply filters on indications that contain embedded instances.

5. The Resolver will validate EmbeddedInstance properties present in class and instance requests and responses sent to the repository.  This was intended to validate the creation and modification of EmbeddedInstance properties during CreateInstance, ModifyInstance, CreateClass, and ModifyClass operations, but has the side effect of validating the EmbeddedInstance properties for all responses returned from the repository as well.  When validating EmbeddedInstance properties, the Resolver will make sure that the class specified by the qualifier exists in the namespace (for class operations), and will ensure that the instance contained in any EmbeddedInstance property is an instance of the class specified by the EmbeddedInstance qualifier.  The embedded instance itself will also be tested to make sure it is well-formed.  If the Object Normalizer is enabled, it will perform the same validation of the EmbeddedInstance properties and parameters present in responses returned by providers.  If the Normalizer is not enabled, then the provider developer is responsible for ensuring that any embedded instances it returns as properties or parameters are well-formed and of a class or subclass of the class specified by the EmbeddedInstance qualifier value for that property or parameter definition.

6. For SNIA interoperability, in Pegasus 2.5 the EmbeddedObject qualifier is supplied on EmbeddedObject properties and parameters if the SNIA interoperability flag is set.  Similarly, the EmbeddedInstance qualifier is required on applicable instance properties and method parameters.  If both the SNIA interoperability flag is set and the Object Normalizer is enabled, the Normalizer will retrieve the qualifier from the class definition and attach it to the EmbeddedInstance property.  If the Normalizer is not enabled, then a provider that requires SNIA interoperability should provide the EmbeddedInstance qualifier as part of any responses that include EmbeddedInstance property or parameter values.  If the provider does not, then the XML encoder will add the EmbeddedInstance qualifier.  In this case, the value of the EmbeddedInstance qualifier on the property or parameter may not match the value of the EmbeddedInstance qualifier in the class definition.  Since the XML encoder does not have class definition information available to it, it cannot retrieve the proper qualifier value from that class definition.  Instead, the XML encoder will use the class name of the instance as the value for the EmbeddedInstance qualifier if it is a non-array property or parameter.  If the property or parameter is an array, then the XML encoder will assume that all the instances in the array are of the same class, and use the first instance's class name to populate the value of the EmbeddedInstance qualifier.  Note that this "best-effort" attempt to supply a legitimate class name is limited by the information available at the time of XML encoding, and it is ultimately the responsibility of the provider developer to ensure that the server is returning correct qualifiers as part of responses generated from a provider.

While specification issues are outstanding regarding the finalization of the attributes used to identify EmbeddedInstances in CIM-XML, the code submitted for this PEP will be surrounded by #ifdef blocks that will allow a developer to include or exclude the functionality.  Currently, the default will be to include the functionality so that a developer will have to set a build-time environment variable to disable EmbeddedInstance support.  If the requisite specifications are not approved by the time 2.6 is released, however, then the default behavior will be changed to disable EmbeddedInstance support by default and require the developer to set a flag to enable the functionality in the build.

ISSUES

1. Currently this PEP does not propose to validate the type of instances within an embedded instances property for all requests, as that could potentially be a very expensive operation, but could be done if the Architecture committee feels it is necessary.  EmbeddedInstance properties created or modified as part of CreateInstance and ModifyInstance operations serviced by the repository will be validated by the Resolver, but providers are responsible for validating any EmbeddedInstance properties or parameters they receive as part of client requests.  At this point it is also assumed that providers will be responsible for sending valid EmbeddedInstances as part of a response or indication.

2. For SNIA Interoperability, the EmbeddedInstance qualifier is required on applicable instance properties.  This is more difficult than embedded objects, where the EmbeddedObject qualifier could be added during XML encoding, because it is a boolean qualifier.  The EmbeddedInstance qualifier needs to be aware of what class the embedded instances belong to. This is potentially problematic for arrays of embedded instances, where it is not clear that the instances are all members of the same class, but could be instances of some common superclass.  One possible solution is that if the EmbeddedInstance qualifier is not present, the XML encoder will assume that the instances in the array are all the same class and if they are not, make it the responsibility of the provider to create the proper qualifier or have it done automatically by the Normalizer (if Normalization is enabled), which can retrieve the qualifier from the class definition.

Rationale

As this is merely an extension to the embedded object support already present in Pegasus, the vast majority of the implementation decisions mirror those made in PEP 194.  The SNIA community will require this functionality to be present in their WBEM infrastructure very soon.  It is critical to have embedded instance support in order for vendors to comply with certain profiles and subprofiles specified by SMI-S v1.1 and beyond (e.g. Job Control, Filesystem Manipulation, Copy Services).  Even if SMI-S v1.1 is not officially released for some time, vendors have to start their implementations and testing of the specification well before it can be released.

Although this PEP is proposed for the Pegasus 2.6 release, given the time requirements within SNIA to have this functionality, embedded instance support needs to be available in a version of Pegasus before the Pegasus 2.6 release date.  Although ideally the Main branch of CVS should be stable, in practice that is not always the case.  The SNIA community, as well as any other users that require embedded instances, will need access to a stable version of Pegasus containing this functionality.  The current plan is to submit the changes required by this proposal early on in the development of Pegasus 2.6 and to create a branch off of the main branch where 2.6 development is taking place if it is sufficiently stable.  If other significant changes have been made to destabilize the main branch, then a branch will be created off of the 2.5 release branch and the required code changes will be resubmitted there.

Schedule

Discussion

(dave_sudlik) Where is the class information associated with the embedded instance (ie. the value of the EmbeddedInstance string qualifier) captured in the CIM-XML? The status of the EmbeddedClassName attribute in CR01600.001 (was Interop-00217.001) is unclear to me. I see it in the CR text, but then below it says it's removed because 'we cannot find a reason for passing the classname information through cimxml'. It sounds like the classname is just an issue for the encoder? This CR can be found at http://www.dmtf.org/membertools/download.php?document_id=11940
(a_dunfey) As the CR owner, I'm sure Karl can address this with more certainty, but I was under the impression that there is no longer an attribute that specifies the class name of the embedded instance. As you pointed out, the discussion section of that CR states that the EmbeddedClassName attribute was removed, and in fact the definition of that attribute was removed from the main body of the CR. There are still some references to the EmbeddedClassName attribute in the CR, but they seem to have been left there accidentally (Karl?), and the CR probably needs to be cleaned up a little bit.

(david_dillard) What are the ramifications of not validing embedded instances?
(a_dunfey) The provider would have to be responsible for validating the types of any embedded instances it receives as part of a request, and it would have to be well-behaved enough to send responses and indications with embedded instance of appropriate classes.
(a_dunfey) As a follow-up to the question and answer above, it may be necessary to do validation of the embedded instance types for instances that are created in the repository.

(david_dillard) Can I suggest a combination of #1 and #2. Make this the first checkin and then make the branch. This would seem to be the least amount of work. If a separate branch is done right when 2.6 starts then people must choose between embedded instances or the other functionality. Nightly builds will have to build both branches to test everything, etc.
(a_dunfey) David, you may suggest anything you want that makes my life easier. That would mean I'd only have to do one checkin. I had anticipated creating a branch so that we wouldn't have to freeze checkins for other developers. That would mean that I would have to merge my code in with whatever changes may have been made between the 2.5 release point and whatever fixes may have gone in in the meantime, but I'm willing to take on whatever extra work that entails if people don't want to freeze the main development branch.
(dave_sudlik) Would this be the 2.5.1 branch?
 (r.kumpf) I would prefer to avoid a community-wide commit synchronization, if possible. I'd favor submitting the EmbeddedInstance support to the main branch and (if necessary) to a development branch at the same time. (Or create the development branch after submittal if the main branch is stable enough at commit time.) Dual maintenance of fixes would be needed in the main branch and the development branch, but this would isolate the development branch from instability in the main branch.
(a.dunfey) I'm definitely OK with doing it that way. Option 3 was suggested as part of the first round of review comments, but if you!sre concerned about freezing the Main branch (which I was initially reluctant to do as well), then I!sm certainly willing to do two checkins if need be.

(r.kumpf) Question: Are EmbeddedInstances "transient", or are they expected to be "queryable"? E.g., could I expect to use the instance name to perform a GetInstance operation?
(a.dunfey) They certainly can be transient, but I don't know that they have to be. In the CIM2.10 MOF, there are many examples of EmbeddedInstances that are instances of type CIM_Error, which are transient, and setting parameters, which are similarly transient (otherwise the methods would just use reference params). That said, the EmbeddedObjects in ModifyInstance indications are presumably persistant (at least for a time), so there!ss no reason why EmbeddedInstances couldn't be persistant in some cases as well.

(r.kumpf) Validation of the type of an EmbeddedInstance property value should probably be consistent with the validation of reference property types in association instances (which I am not sure Pegasus currently does). That was probably a confusing statement, so I will try again. Whatever level of validation Pegasus does to ensure that the type of a reference property value is consistent with the class definition, that same level of validation should be done on the type of an EmbeddedInstance property value.
(a.dunfey) That sounds reasonable. I!sll investigate and update this proposal accordingly.

(r.kumpf) Regarding the preceding paragraph, what is a "valid EmbeddedInstance"?
(a.dunfey) That's a good question. In the context of the statement I made, I meant that it was an instance of the class specified in the qualifier (or subclass thereof), didn't invent properties that aren't part of the instance!ss class definition, the values of the properties match the property definitions in the instance!ss class, etc. There are potentially other considerations, such as which properties must be present. I haven't found any pertinent statements regarding EmbeddedInstances and EmbeddedObjects in general. I imagine that the remaining tests for "validity" are implementation-, use-case-, or class-specific. For example, in the CIM2.10 MOF, the CIM_ExportedFileShareSetting class is used as an EmbeddedInstance parameter, but the description qualifier on the class gives specific rules as to the presence of properties and the meaning of a NULL value for a property.

(r.kumpf) Question: Is it possible to declare an EmbeddedInstance qualifier that is not tied to a specific class type? I expect that the answer is "no", and that there is no such thing as an EmbeddedInstance property that is not strongly typed. Is this clarified in the spec?
(a.dunfey) You!sre correct that the answer is !sno!s. The strong typing of the EmbeddedInstance qualifier is what really makes it distinct from the EmbeddedObject qualifier. I believe this distinction is clear in the CIM Specification, and it also includes language stating that the value of the EmbeddedInstance qualifier must not be null.

(r.kumpf) Validation of the type of an EmbeddedInstance property value should probably be consistent with the validation of reference property types in association instances (which I am not sure Pegasus currently does). That was probably a confusing statement, so I will try again. Whatever level of validation Pegasus does to ensure that the type of a reference property value is consistent with the class definition, that same level of validation should be done on the type of an EmbeddedInstance property value.
(a.dunfey) That sounds reasonable. I'll investigate and update this proposal accordingly
UPDATE: If the Normalizer is not enabled, I did not find any place where we currently do type checking on reference properties. As a result, if the Normalizer is not enabled, the type will not be validated for Provider responses.

Copyright (c) 2005 EMC Corporation; Hewlett-Packard Development Company, L.P.; IBM Corp.; The Open Group; VERITAS Software Corporation

Permission is hereby granted, free of charge, to any person obtaining a copy  of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

THE ABOVE COPYRIGHT NOTICE AND THIS PERMISSION NOTICE SHALL BE INCLUDED IN ALL COPIES OR SUBSTANTIAL PORTIONS OF THE SOFTWARE. THE SOFTWARE IS PROVIDED  "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Template last modified: March 9th 2004 by Martin Kirk
Template version: 1.8