Pegasus Enhancement Proposal (PEP)

PEP#: 361

PEP Type: Functional

Title: CIMCLI Embedded Instance support

Status: draft

Version History:

Version	Date	Author	Change Description
0.9	10 March 2011	K. Schopmeyer	Initial Submission
1.0	25 June 2011	K. Schopmeyer	Extend and clarify: --More examples --Define invoke method functionality --Note that we have to support arrays and define format.
1.1	4 Aug 2011	K. Schopmeyer	Changes from Arch team review 4 Aug. 2011 -- Add implementation details section --Correct errors noted by other comments -- Modify the embedded instance start token from "{" to either "={" or "{" -- Extend the formatting of array of embedded instances to be either repeated input of the parameter or comma separated parameters --Clean up some spelling and text

Abstract: Proposal for extensions to CIMCLI to properly use of embedded instances and embedded objects in CIM operation requests and responses

Definition of the Problem

cimcli is a command line tool to execute cim operations (today using the CIM/XML protocol) from parameters on the command line and to display responses to these operation requests from a CIM server.

Today cimcli can execute all of the operations defined in the DMTF CIM/XML operations (ex. getInstance, enumerateInstances, associators, references, createInstance modifyInstance, InvokeMethod) and display the responses returned by the CIMServer in several different formats (XML, MOF, etc.).

Today cimcli has limited support for embedded instance display on output and NO support for creation/modification of objects with embedded instances. In effect, it presents most of the raw information provided by the server but provides not special processing of returned intances or objects that contain embedded instances. Further, it cannot create an embedded instance for use with createInstance, modifyInstance, or within an input parameter.

cimcli issues include with embedded instances include:

1. Returned result of instance operations displays do not format these objects but simply display them in the returned XML format.
2. There is no way to create or modify instances with properties that represent embedded instances or objects.
3. There is no way to define invoke method parameters that include embedded instances or objects.
4. Returned parameters display embedded objects as strings with the string attribute completely ignoring any indication that the returned objects are embedded CIM Instances.

Proposed Solution

The CIM metamodel allows the concept of instances/objects embedded within other instances. While today this is used largely in indications the capability is more general and can be used within all classes so that embedded instance could be component of any of the instance operations (getInstance, enumerateInstances, associators, references, InvokeMethod).

An instance of class that contains another embedded instance actually includes embedded instance within the embedding instance rather than a reference to an instance (i.e. pass by value). As an example an embedded class and embedding class could be defined as follows:

// Class to embed in
      TST_CLITestEmbedding

class TST_CLIEmbedded

      {

           [key] uint32 id;

           string name;

           datetime date;

      };

      

      // Class that defines an embedded instance

      class TST_CLITestEmbedding

      {

           [key] uint32 id;

           string name;

           [EmbeddedInstance("TST_CLIEmbedded")]

           string embeddedInst;

      };

The above defines the class TST_CLIEmbedded as a separate class but expects instances of TST_CLIEmbedded to be inserted in the property embeddedInst of the class TST_CLITestEmbedding. An operation that returns instances of TST_CLITestEmbedding class is expected to include instances of TST_CLIEmbedded as the value of the property embeddedInst. However within the CIM model as defined by DSP0004 there is no concept of a data type to represent these embedded object (i.e. no CIMObject or CIMInstance as CIM data types). Thus, to express the definition of a property as an embedded instance the class definition of TST_CLITestEmbedding must define the property as a string and clarify that this is an embedded instance property with either the qualifier EmbeddedObject or EmbeddedInstance as shown in the above example.

These two qualifiers differ in that:

EmbeddedObject qualifier would allow either class or instance definitions but places no further limit on the class to be represented.
The EmbeddedInstance qualifier includes an optional value which is the name of the class to be embedded.

Since this must all be passed over protocols between the client and the server and it is not required that qualifiers be part of the information provided with instances, at least the CIM/XML protocol has been extended to include an attribute for String properties that contain embedded instances so that they can be identified by the parser without requiring access to the class itself. This is the EmbeddedObject attribute (DMTF spec. DSP0201) which allows the protocol to define whether the property is defiened by the EmbeddedInstance or EmbeddedObject qualifiers. This does not allow passage of the optional value that can be part of an EmbeddedInstance qualifier but does allow the recieving parser to determine if a string represents an embedded object or instance. Also, the CIM/XML protocol defines the format required for embedded objects/instances as the CIM/XML representation of the object.

The Pegasus environment has been extended to include new data types for CIMObject and CIMInstance and internally the server handles the conversion of any qualifying string to these data types so that internally in the Pegasus server the concept of embedded instances does exist. On the client side, the Pegasus infrastructure (src/Pegasus/Clients) automatically maps any property received in a response with an EmbeddedObject attribute back to a CIMInstance. However, today it does not do this with parameters returned from invokeMethod.

To extend the support for working with embedded objects we propose to extend cimcli as defined in the following sections:

Format of output of instances/objects received in operation responses.

cimcli attempts to display the results of operations including any instances/objects received. It provides multiple output formats but primarilly MOF and XML today.

XML formatted output that attempts to show the CIM/XML format for the object,
MOF formatted output that maps the object to the DMTF defined MOF format.

Extend support for display of responses received by cimcli that include embedded instances/objects to actually display the embedded entities as MOF for MOF output. Today what is presented to the user is just the value of the String property/parameter that is an embedded entitity containing the CIM/XML XML representing the object. This is a) fairly incomprehensible, b) no assurance that it represents a viable object.

What the user sees today on the output of a getInstance operation that included instance would be similar to the example below:

RESPONSE: in MOF format (without qualifiers) (REQUEST Operation: cimcli ei TST_TestEmbedding)

instance of TST_TestEmbedding

      {

      key = 1;

      name="fred";

      embeddedInst =

      "<INSTANCE CLASSNAME=\"Test_CLITestEmbedded\"
      >\n<QUALIFIER NAME=\"Version\" "

          "TYPE=\"string\" TOSUBCLASS=\"false\"
      TRANSLATABLE=\"true\">\n<VALUE>2.5.0</VALUE>\n</QUALIFIER>\n<QUALIFIER


      "

          "NAME=\"Description\" TYPE=\"string\"
      TRANSLATABLE=\"true\">\n<VALUE>Class "

          "to test handling of embedded instances and
      objects by cimcli. This "

          "class is to be embedded but cannot be directly
      accessed(enumerated, "

         
      "etc.)</VALUE>\n</QUALIFIER>\n<PROPERTY
      NAME=\"key\"  CLASSORIGIN=\"Test_CLITestEmbedded1\" "

          "TYPE=\"uint32\">\n<QUALIFIER
      NAME=\"Key\" TYPE=\"boolean\"
      OVERRIDABLE=\"false\">\n<VALUE>TRUE</VALUE>\n</QUALIFIER>\n<VALUE>100</VALUE>\n</PROPERTY>\n<PROPERTY

      "

          "NAME=\"name\" 
      CLASSORIGIN=\"Test_CLITestEmbedded1\"
TYPE=\"string\">\n<VALUE>Ronald</VALUE>\n</PROPERTY>\n</INSTANCE>\n";

      };

Note that there has already been a conversion in the client infrastructure in that the value of embedded as the CIM/XML protocol sees it is escapes XML escape characters (ex. $lt) and these have been mapped back to their real characters (i.e.<).:

RESPONSE: in XML format: ((REQUEST Operation: cimcli ei TST_TestEmbedding -x)

<INSTANCE
      CLASSNAME="Test_CLITestEmbeddedClass" >

      <PROPERTY NAME="key"  TYPE="uint32">

      <VALUE>1</VALUE>

      </PROPERTY>

      <PROPERTY NAME="embeddedInst"  TYPE="string"
      EmbeddedObject="instance" EMBEDDEDOBJECT="instance">

      <VALUE>&lt;INSTANCE
      CLASSNAME=&quot;Test_CLITestEmbeddedInst&quot;
      &gt;&#10;&lt;QUALIFIER
      NAME=&quot;Version&quot; TYPE=&quot;string&quot;
      TOSUBCLASS=&quot;false&quot;
      TRANSLATABLE=&quot;true&quot;&gt;&#10;&lt;VALUE&gt;2.5.0&lt;/VALUE&gt;&#10;&lt;/QUALIFIER&gt;&#10;&lt;QUALIFIER
NAME=&quot;Description&quot;

      TYPE=&quot;string&quot;
      TRANSLATABLE=&quot;true&quot;&gt;&#10;&lt;VALUE&gt;Class
to

      test handling of embedded instances and objects by cimcli. This
      class is to be embedded but cannot be directly
      accessed(enumerated,
      etc.)&lt;/VALUE&gt;&#10;&lt;/QUALIFIER&gt;&#10;&lt;PROPERTY
NAME=&quot;key&quot;
&#32;CLASSORIGIN=&quot;Test_CLITestEmbedded1&quot;
TYPE=&quot;uint32&quot;&gt;&#10;&lt;QUALIFIER
NAME=&quot;Key&quot;

      TYPE=&quot;boolean&quot;
      OVERRIDABLE=&quot;false&quot;&gt;&#10;&lt;VALUE&gt;TRUE&lt;/VALUE&gt;&#10;&lt;/QUALIFIER&gt;&#10;&lt;VALUE&gt;100&lt;/VALUE&gt;&#10;&lt;/PROPERTY&gt;&#10;&lt;PROPERTY

      NAME=&quot;name&quot;
      &#32;CLASSORIGIN=&quot;Test_CLITestEmbedded1&quot;
TYPE=&quot;string&quot;&gt;&#10;&lt;VALUE&gt;Ronald&lt;/VALUE&gt;&#10;&lt;/PROPERTY&gt;&#10;&lt;/INSTANCE&gt;&#10;</VALUE>

      </PROPERTY>

      </INSTANCE>

Since the goal of the XML output format is to represent the XML definition of the received entities as closely as possible presenting the XML string for embedded instance properties is correct. This will not be modified.

Thus, the user will be able to see:

XML output of the response object including the value of embeddedInstance properties in the CIM/XML format.
MOF output of the response object including the value of embeddedInstance properties in MOF format.

Extending the MOF outptut format includes at least the following functionality to be able to display the MOF output for embedded objects:

Extend the MOF formatting capability to recognize properties and parameters that qualify as embedded entities. This is automatic with properties in responses in that they are converted back to CIMInstance by the infrastructure. With parameters returned from invokeMethod cimcli must guess today since the Pegasus client infrastructure does not convert them.
Display the embedded instances including the classname and properties in the mof format as instances but embedded within other instances. Note that there is no "offical" defintion of an embedded instance syntax in mof today for CIM V2. Therefore, the output format from cimcli will look like mof but is probably not compilable by most mof compilers.
Error handling for those cases where the string value cannot be parsed back to an instance.

Note that this capability will be general to all output display including responses to instance requests (getInstance, enumerateInstances, references, associators, invokeMethod) except for the getProperty which a) has been deprecated and b) does not carry any typing information in the protocols so that the receiving parser has no knowledge at all about the value except that it is a string of characters.

An example of what the output of the MOF format is shown below, in this case including qualifiers (which are normally optional):

[Version ("2.5.0") : Restricted, Translatable, 

      Description ("Class to test
        handling of embedded instances and "

      objects by cimcli. This class
        embeds objects and instances")]

      instance of
        Test_CLITestEmbeddedClass

      {

         [key]

          key = 1;

         
        [EmbeddedInstance ("Test_CLITestEmbedded")]

          embeddedInst =
        instance of Test_CLITestEmbedded

             
        [Version ("2.5.0") : Restricted, Translatable,

        
             Description ("Class to test handling of
        embedded instances "

             
        "and objects by cimcli. This class is to be embedded but "

             
        "cannot be directly accessed(enumerated, etc.).")]

             
        instance of Test_CLITestEmbedded1]

          {

            

        [key]

        
             key = 100;

        
             name = "Ronald";

          };

      };

or without qualifiers:

instance of
        Test_CLITestEmbeddedClass

 {

         

        key = 1;

         

        embeddedInst = instance
        of Test_CLITestEmbedded

          {

        

             key = 100;

        
             name = "Ronald";

          };

      };

Note in the example above that the EmbeddedInstance qualifier is included in this output. Since there is some impreciseness in the definition of embedded entities in MOF (i. e. it is not really defined in the DMTF specification DSP0004 we have some freedom in syntax but in general it appears as the most logical MOF representation of the embedded instance possible. Note also that since MOF does not include CIM type information as part of the instance MOF the fact that the CIM type for the embeddedInst property is a Pegasus Internal type (CIM Instance) whereas the received type is string is not visible in the output.

The corresponding output in XML format would be generally as follows:

path= Test_CLITestEmbeddedClass.Id=101
<INSTANCE CLASSNAME="Test_CLITestEmbeddedClass" >
<PROPERTY NAME="Id" TYPE="string">
<VALUE>101</VALUE>
</PROPERTY>
<PROPERTY NAME="embeddedInst" TYPE="string" EmbeddedObject="instance" EMBEDDEDOBJECT="instance">
<VALUE><INSTANCE CLASSNAME="Test_CLITestEmbedded1" >
<QUALIFIER NAME="Version" TYPE="string" TOSUBCLASS="false" TRANSLATABLE="true">
<VALUE>2.5.0</VALUE>
</QUALIFIER>
<QUALIFIER NAME="Description" TYPE="string" TRANSLATABLE="true">
<VALUE>Class to test handling of embedded instances and objects by cimcli. This class is to be embedded but cannot be directly accessed(enumerated, etc.)</VALUE>
</QUALIFIER>
<PROPERTY NAME="Id"  CLASSORIGIN="Test_CLITestEmbedded1" TYPE="string">
<QUALIFIER NAME="Key" TYPE="boolean" OVERRIDABLE="false">
<VALUE>TRUE</VALUE>
</QUALIFIER>
<VALUE>100</VALUE>
</PROPERTY>
<PROPERTY NAME="name"  CLASSORIGIN="Test_CLITestEmbedded1" TYPE="string">
<VALUE>Ronald</VALUE>
</PROPERTY>
<PROPERTY NAME="comment"  CLASSORIGIN="Test_CLITestEmbedded1" TYPE="string">
<VALUE>Instance created by provider.</VALUE>
</PROPERTY>
</INSTANCE>
</VALUE>
</PROPERTY>
<PROPERTY NAME="embeddedObj" TYPE="string">
</PROPERTY>
<PROPERTY NAME="comment" TYPE="string">
</PROPERTY>
</INSTANCE>

Extensions for Definition of Embedded Instances

Extension of the creation of instances and modification of instance operations so that they can contain embedded instances. Today cimcli allows execution of creation/modification operations from the console with definition of the instances (i.e. properties and values) defined on the command line. It does not allow the definition of properties that are embedded instances of the created/modifed instance except as string properties that define the XML string representing the instance.

The format for input of information for creation/ modification of instances today in cimcli is based on using name/value pairs for the properties to be included in the created instance where the names are property names and the values are the corresponding property values. Thus, the command line for the example class TST_CLITestEmbeddedClass would be:

cimcli ci TST_CLITestEmbeddedClassid=1 name=fred ...

without the embeddedInst property.

For this operation cimcli does the following:

Get the class for TST_CLITestEmbedding from the server so that we have the types for the various properties. This is required because we do not know what properties exist and thir CIM Types without the metadata represented by the class definition.
Parse each name/value pair on the command line to try to create the value with the type for the class. This assures that the value information input for each property matches the CIMType for that property and creates the Pegasus instance for the class TST_TestEmbedding. Thus, for example, if the input included id=blah cimcli should generate an error since this is not a valid uint32 value.
Execute the createInstance operation with the created class against the defined server.
Display the response which in the case of create instance would be either a good response with a CIM Object Path for the instance created in the server or an exception if the operation failed.

A user with real courage could actually try to encode the the embeddedInst property value but since it is full of special characters with all of the character exceptions for most shell programs that could be an exercise in futility not to mention the requirement to enter a very long string correctly.

With the embedded instance extension the user should be able to enter the definition as follows:

cimcli ci TSTCLITestEmbedding id=1 name=fred embeddedInst={TST_Embedded id=3 name=jones }

Thus we have embedded the definition of the embedded instance within the definition of the embeddedInst property. The "={" is a token for the parser that what follows in an embedded instance. The"}" is the terminator for definition of properties within the embedded instance.

The change to the cimcli command line is as follows:

Recognize the tokens "={" or "{" when parsing the name/value pairs which initiates a recursive building of an instance. Note that this token actually appears as the last characters of a property name definition with no white space. This was done as a convenience for the parsing since the name/value pair then appears as a single parameter in the list of input parameters.
Recognize the class name as the first parameter after "{" or the tail of the parameter containing the property name terminated by "{".
Recognize the "}" parameter as the indication to end the recursive build of an instance.
Parse everything within the "={" to "}" as classname and properties of the embedded instance rather than the parent instance.
Create the embedded instance and insert it as the property value in the parent instance.

NOTE: We reserve the right to change the "{" and "}" characters to provide that character pair that represents enclosing something but gives the easiest implementable solution for a wide range of command shells(bash, sh, windows cmd.exe, etc.) so that the no special characters are used on any platform. There may be several minor changes during the development to adapt this general structure to the vulgarities of the different command shells used (ex. sh, bash, windows command.exe, etc.)

The extensions to cimcli to accomplish this include:

Extend the syntax so that the "={", {" and "}" are recognized as special parameters on the command line as well as the class name as first parameter after the "={".
Extend the object creation facility (objectBuilder class) to that it is capable of recursively building instances and of building the property/parameter with the recursive instance value in the property.

Extensions for Display and Definition of Parameters for InvokeMethod

Extension of parameters on invoked to include the same capabilities to define and display. Since the capabilities for displaying and creating embedded instances defined above are general, they will apply to parameters as well as properties.

Thus, a cimcli input for an invoke method on a class with the following method:

    uint32 embeddedInstParamMethod(
        [In, Out] string arg1,

        [EmbeddedInstance("Test_CLITestEmbedded1"), In, Out] string arg2,

        [In, Out] string arg3);

would look generally like.

    cimcli im -n test/TestProvider Test_CLITestEmbeddedClass \
        embeddedInstParamMethod arg1="test string" \
        arg2{Test_CLITestEmbedded1 Id=501 }

This defines the method embeddedInstParamMethod with the following arguments:
1. arg1, string with value "test string"
2. arg2 embeddedInstance with class name = Test_CLITestEmbedded1 and property Id with value 501.

and the response to this operation (assuming that the provider simply returned the input parameters) would look like:

Return Value= 0
string arg1=test string
instance arg2=
path= Test_CLITestEmbedded1

    instance of Test_CLITestEmbedded1
    {
      Id = "501";
    };

Note that the presentation of the instance information includes path information only because that is the standard output format in cimcli.

Other general requirements include:

1. Account for arrays of embedded instances and embedded objects

This will be accomplished without modifying the above format by allowing multiple definitions of the same property/parameter name. For
a non-array property/parameter this is illegal. But for an array, each additional definition on the command line is an new element in the array.

Thus, for example:

If the class is defined as:
    class Test_X
    {
        [Key] string Id;

        [EmbeddedInstance("Test_CLITestEmbedded1"),
        Description("Property containing embedded Instance")]
        string embeddedInst[];

        uint32 intArray[];
    };

Today cimcli handles input of array parameters with a format that separates the items of the array with "," and no spaces. Thus input of a
create instance would be formatted as follows:

    cimcli ci Test_X intArray=3,9,24,7

or a second form where the parameter is repeated with each new value to add to the array as follows:

    cimcli ci Id=3 intArray=3 intArray=9 intArray=24

The goal is to extend this paradigm as much as possible to the creation of arrays of embedded
instances

The input would appear as follows:

cimcli ci TST_X id=1 name=fred embeddedInst{TST_Embedded id=3 name=jones } \                               embeddedInst{TST_Embedded id=4 name=Smith date=now } or cimcli ci TST_X id=1 name=fred embeddedInst{TST_Embedded id=3 name=jones }, \                                                                              {TST_Embedded id=4 name=Fre }While this may seem like a strange format, it is actually one of the allowed formats today for array input in cimcli for array values and also
much simpler than trying to create an additona concept of embedding for someting like arrays of embedded instances that is rarely used.

2. Handling recursive embedded instances

We must allow recursion of embedded instances/objects where an embedded instance contains a property that is another embedded
instance. Thus, we must allow, for example recursive embedding as shown below with the class Test_Y;

class Test_Y
{
        [EmbeddedInstance("Test_Y")]
    string embeddedInst;
};

class Test_X
{
   [Key], uint32 Id;
        [EmbeddedInstance("Test_Y")]
    string embeddedInst;
};

The ciimcli create instance input for this would generally be:

     cimcli ci Test_X Id=3 emmbeddedInst={Test_Y embeddedInst={Test_Y } }

Where we are defining an instance of Test_X with key=3 and an embedded instance property that contains another embedded
instance.

NOTE: It is not clear that Pegasus and its providers/repository handles this today but we will provide the capability to
generate and display this form in cimcli in any case. This PEP does not propose to fix any issues in Pegasus server itself.

Class Operations (getClass, enumerateClasses, associators, references)

A number of the CIM operations return classes. However, this PEP does not include any changes to the class operation output formatting. The manner in which classes are viewed is correct today and cimcli has no capability to create new classes from the console.

Other Changes

Add tests for these extensions to the cmcli test environment.
Extend the documentation to include this functionality and examples (internal help, msg files, cimcli/doc).
Extend CIMCLI so that it can test for error code returns (parameter such as --expexit 3, so cimcli would output success code only if that return code received). This would allow Makefiles to include tests for specific responses from the server and extension of automated tests within cimcli test makefile (src/Providers/TestProviders/CLITestProvider/tests).

Pegasus Components Affected

cimcli files (src/Clients/cimcli)

Extend objectBuilder class to handle input of embedded instances. Today in only handles input of scalar or array parameters of other types.
Correct issues with CIMCLIOutput to output MOF for embedded instances received as properties or output parameters. Today it outputs the XML formatted string for embedded instance properties.
Add new input parameter --expexit to allow cimcli to test for particular exit codes on exit.
Update help and doc to reflect new functions.

CLITestProvider and CLI test files(src/providers/TestProviders)

Add a set of classes to the CLI test mof to simplify test of embedded instance properties and parameters. (Load/CLITestProvider.mof and corresponding provider registration)
Extend CLITestProvider to handle classes above. ( CLITestProvider/CLITestProvider.cpp)
Add tests to the CLI Test makefile (CLITestProvider/tests/Makefile)

The MOF Formatter(src/Pegasus/General/MOFWriter.*)

There may be changes to the MOF formatter primarily to correct some existing errors. NOTE: Today the MOFWriter does a poor job of formatting mof output.
It formats as one line per property and there is a reformatter in cimcli. That should be changed in the future but not for this proposal.

Rationale

This proposal was motivated in part by a request from SNIA to provide more complete support for embedded entities in cimcli to support testing

Schedule

Implementation available August 2011 for incorporation in Pegasus 2.12 with the changes submitted through bug 9007.

Discussion

Permission is hereby granted, free of charge, to any person obtaining a copy of

this software and associated documentation files (the "Software"), to deal in

the Software without restriction, including without limitation the rights to

use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of

the Software, and to permit persons to whom the Software is furnished to do so,

subject to the following conditions:

The above copyright notice and this permission notice shall be included in all

copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR

IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS

FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR

IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN

CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Template last modified: February 17th 2009 by Martin Kirk
Template version: 1.15