Pegasus Enhancement Proposal (PEP)

PEP #: 283
PEP Type: Functional
Title: Enhancing CLI
Created: 10 Oct. 2006
Authors: K. Schopmeyer
Status:  draft

 Version History:

Version Date Author Change Description
1.0 20 Oct 06  ks Initial Submission
 1.1 27 Oct 06  ks   Respond to comments from cv1
1.2 Jan 2008 ks Rewrite proposal from earlier comments
1.327 Oct 2008ksheavy rewrite and cleanup
1.4Nov 2008ksComments from Version 1.3
1.512 Aug 09ksFurther Cleanup from 1.3.  The i.4 version was never reviewed. This version approved by Arch team

 

Abstract: Add selected Enhancements to cimcli to complete implementation of some of the basic operations and add a new test operation


Note: Subsequent to the approval, one new feature was added, an option to sort the output (--sort). This feature is not documented in this note other than this coment. The code in this PEP was incorporated in Pegasus 2.10.0

1. Definition of the Problem

CLI  (now cimcli) has been used as a test tool for Pegasus for several years now.  However it has several limitations.  This PEP proposes some updates to CLI to complete the ability to execute the basic operations as much as possible with a command line tool.

2. Proposed Solution

We propose changes to a number of the operation while keeping the existing behavior as much as possible so that we do minimal damage to existing scripts and Makefiles that use cimcli.

The changes are:

  1. Complete Instance Creation function.
  2. Complete Instance Modification function.
  3. Complete the InvokeMethod so we can handle a more complete set of parameters than we do today.
  4. Modify those operations that use object path as input (getInstance, etc.) so that there is an alternative form of the input using the same name/value pair mechanism as the instance creation and modification.  This means that
  5. the user has a somewhat simpler input form that having to exactly recreate the CIMObjectPath with its required quotation marks.  The existing form would be kept.
  6. Add a new function testInstance that would be equivalent to createInstance except that it would build the defined instance in the cimcli client and compare it for equality of defined properties with the same CIMObjectPath in the
  7. server (i.e. get the instance from the server and compare).  This would allow some value testing to be accomplished by cimcli which we cannot do today.

2.1 Complete the Instance Creation Function

Complete the instance creation  function so that correct and complete instances with all possible property types can be input to create an instance. This extension will allow creating/modifying instances through CLI with all possible property types (except embedded instance) and with the possibility to build both complete instances and instances with selected properties.

The proposal is to define a simple format for input based NVPN (NVPN - Name/Value Pair notation)  with the ability to handle all CIM value types (with the exception of embedded instance) and that will accept both scalars and arrays.  This would allow CLI to create just many legal CIM instances. The goal of this form would be to define a format that is simple for the user to input information from a shell or script and minimized the use of special shell characters. The inputs would NOT include the type information; this will be acquired from the class.  Note that there is an FUTURE option to define the type with the NVPN which could be used to create effectively illegal instances; however this is not yet implemented. 

Thus, a typical create instance might look like:

        cimcli ci PG_myClass name=fred age=30 arrayvar=abc,def,ghi

        cimcli ci myClass name="my name" age=30 arrayvar=abc,"second item",ghi

                    cimcli ci myClass name="fred", age= arrayvar=abc,def,ghi

                In the second case age is created as a property with NULL value.

Syntax for the name/value pair command line:

NOTE: The following syntax definition is based on what would be seen by cimcli internally from the command line, not what the user inputs to the command line. This is because we must support multiple shells and the special characters differs between different shells:

Thus, the syntax for the createInstance operation (as seen by cimcli after any specific processing by the command line processor) would be:

    cimcli ci <class_name> <featureValueDefinition>*

The syntax for property definition is:

    featureValueDefinition = <featureNameDefinition> <nameValueSeparator>  [value_set]

    nameValueSeparator = "=" | | "!" 

    featureNameDef = [typeDefinition] <featureName>

    FUTURE typeDefinition = "[" <DMTFDataTypeName> "]"

             ISSUE: The [ ] is probably a poor choice of marker characters because shells use them.


    DMTFDataTypeName = "sint8" | "uint8" | "sint8 | "uint16" | "sint16" | "uint32" | "sint32" | "uint64" | "sint64" |

                                             "real32" |  "real64" | "boolean" | "string" | "datetime" | "ref"

    featureName =  <validCIMPropertyOrParameterName>

                               NOTE: See DMTF specs for exact definition of property or parameter name.

    validCIMPropertyOrParameterName =  Any valid CIM Name as accepted by PEGASUS CIMName()

    valueSet = <valueConstant> | <arrayValueConstant>

    valueConstant = <integerValueConstant> | <realValueConstant>  | <stringValueConstant> |

                                              <booleanValueConstant>

 
    integerValueConstant = <binaryValueConstant> | <octalValueConstant> | <decimalValueConstant> |    
                                             <hexValueConstant>

    realValueConstant = [ "+" | "-" ] *decimalDigit "." 1*decimalDigit [ ( "e" | "E" ) [ "+" | "-" ] 1*decimalDigit ]

    booleanValueConstant = "TRUE" | "FALSE" | "true" | "false"

    arrayValueConstant = <valueConstant> * [ "," <valueConstant>]

    decimalValueConstant = [sign] <unsignedInteger>

    octalValueConstant =  [ sign ] "0" 1*octalDigit

    hexValueConstant = [ sign ] ( "0x" | "0X" ) 1*hexDigit

    binaryValueConstant = [ sign ] 1*binaryDigit ( "b" | "B" )

    binaryDigit = "0" | "1"

    octalDigit = "0" | "1" | "2" | "3" | "4" | "5" | "6" | "7"

    hexValue = [sign] ( "0x" | "0X" ) 1*hexDigit

    hexDigit = decimalDigit | "a" | "A" | "b" | "B" | "c" | "C" | "d" | "D" | "e" | "E" | "f" | "F"

    unsignedInteger  = decimalDigit *decimaldigit

    stringValueConstant = 1*( *stringChar  )

   
    stringChar =       "\" """ |           // encoding for double-quote
                                "\" "\" |           // encoding for backslash
                                "\" ","  |          // encoding for comma
                                any UCS-2 character but """ or "\"

    decimalDigit :== "0" ... "9"

    sign ::= "+" | "-"

NOTE: We are trying to make these definitions consistent with the DMTF specs and with the implementation in Pegasus StringConversion.h but with modifications as required to simplify input from a shell with the peculiarities of particular shells. Note also that the name/value pair form is based on NO SPACES between name and value.

The following are examples of name/value pairs that would be legal.

Input of NULL  value properties or properties where the class default value are to be used are represented with the name/value pair but ending with the = sign without a value component as shown below

<Property_Name>=

Thus, for example a class with property age where the intention is to create an instance with age NULL would be:

    cimcli ci PG_myClass name=fred age= size=3 

         ## generate instance with null value for age

NOTE: We elected to do this rather than the keyword NULL to simplify input.  In MOF, all strings are surrounded by quotation marks and one goal here is to be more flexible and avoid the quotation marks as much as possible (they have to be escaped in many shells to avoid them being treated as a special character) so our goal is to avoid the use of quotation marks as a required character for string input.

There is a special case for String properties in that they have two special boundary cases, a) NULL and b) empty which are significantly different.  In order to represent both we need something both since the general form above could server for either empty or NULL. Simply using a form such as cimcli ck PG_myclass name="" is unacceptable since the processing for quotations is shell dependent and we will need some form that can be represented by multiple shells. Therefore we propose that a special name/value separator ("!" rather than "=") be used as the indicator of an empty string.  Thus the form for an empty string parameter would be name! and for a NULL string property would be name=. 

A CIM DateTime value is input as a string with the CIM formatting for ascii date time representation. Note that we have expanded this to allow input of some keywords such as "now" that would force cimcli to provide the current time rather than a fixed time input.  We are providing the extra definitions such as now (The only one implemented to date) as a way that the user can provide date time input without having to go through the hassle of creating the complete datetime string input.

String is special in that it must allow for both NULL value and empty string.  The NULL value is represented with no value component.  An empty string would be input with an empty string value (name="").

This allows inputting property value information with a minimum of extra characters.  For example, quotation marks are only required for string properties that invoke spaces. The quotation mark would be treated as a special character and would require escaping if it was to be included as part of the input string.  The name/value pair separator would be space (the ShellArguementTerminator).  Thus the special characters are:

This will allow inputting all property types defined for CIM with a minimum of extra string enclosing markers such as quotes except that reference properties will be required to be defined in the format for CIMObjectPath strings

NOTE: This differs from the standard MOF input in the limitation on the use of spaces and also the lack of the use of the required { and } to initiate and terminate an array value set. We propose to make the use of the array envelop characters ( { and } ) optional.  In some shells the { } characters are an expansion characters (ex. in bash a={abc,def} expands to a=abc a=def so that we have to be very careful of the use of this character.

String properties represent some unique issues to assure that we can build strings properly, that we can represent NULL,empty strings, and strings that include the argument terminator characters; and finally that we do not have to wrap the strings with enclosing marks unless absolutely required for the parser to separate out the string. One of the goals of this proposal is to make it easy and protective to input to CLI.  Note that in the past one of the continuing issues in using this type of tool was the complexity of inputting information. We often wasted time in testing creating input that was acceptable to the CLI and Pegasus environment instead of testing.  

Behavior:

The cimcli input processing behavior will be that the instance will be created with just the properties  defined as part of the input using the property/value pairs on the input line.  No properties that are not defined in as part of the input will be added to the created instance.   CIM typing for properties will be derived by accessing the CIMClass for property type, array vs. non-array characteristics, etc.

This proposal will NOT allow inputting properties that are not defined in the class or defining the types for properties.

cimcli will acquire the class and will build values for the properties that are defined on the input line. 

Upon successful completion of the createInstance() operation, cimcli will output the returned path.

ISSUE: We do not have a form to allow for fixed size arrays.

Limitations:  

a. No means to specify embedded instances.

2.2 Complete the modifyInstance Function

Add the modifyInstance function this function is not implemented in the existing version of cimcli.  The command is the same format as the createInstance except for the following:

modifyInstance includes the possibility of a propertyList as an input parameter. This simply means that the user can use the -pl option to input a property list

modifyInstance requires that namedInstance (instance with path) be used in place of instance on input. The name component is used to locate the instance to be modified.  To allow for this we have two possibilities, a) build the path from an object name input or build a complete instance with key properties and build the path from that instance.

Syntax:

    cimcli mi <className> | <objectName> *<property_value_definition> [-p <property_list>] [other options]

Behavior:

The behavior of this function will be similar to the behavior of create instance except that it honors the property list option.   That option determines the properties that are requested and which may be modified.  The property list provided on input will be  submitted with the request.  However, it will not be used in the creation of the instance to be submitted with the modifyInstance operation. That will be created from the input parameters.

Thus the algorithm for this operation is:

  1. Get the target class or object name input
  2. Parse the input parameters for property/value pairs
  3. Create instance from input parameters and class definition, This is the modified instance.
  4. If the input included an object name, insert this name into the path component of the modified instance. If it was just the class name, attempt to build the instance name from the instance we built. If the input was only the class name and the interactive flag is set, execute the selectInstance() function to present the user with a list of instances that exist for the target class.
  5. Submit modifyInstance operation with created instance and (if input) propertyList.

Limitations and Issues:

  1. There is no capability to use the include qualifiers capability or to include qualifiers to be inserted into the modified instance.

2.3 Complete InvokeMethod Function

Currently invokeMethod handles a limited set of input types (string, array of strings, and reference) as follows:
  1. The string is defined by the simple keyword=string pair.  No matter what the required value in the class definition, it is encoded as a string
  2. Array of strings is handled by surrounding a set of values in a keyword=value set by {}
  3. A reference is identified by surrounding the text definition of a keyword=string argument with [] to identify it as a  reference.  cimcli then decodes this as a CIMObjectPath.

There is no checking against the invokeMethod definition for correct typing

This PEP proposes to modify that so that we can handle all types of input parameters and that we perform type checking against the method definition in the class.

1. Extend the same form for inputting value information to the CIM parameters of invoke method.  This will allow all of the standard data types to be input except for embedded instance.
2. Change so that the typing is derived from the class definition. Then we would not need the [ or { to distinguish types since the class defines the input types.
2. Allow the current syntax to continue. Since the array form is a subset of what we would allow there is no difference and we can also allow the [ ref marker as an option.

2.4 CIMObjectPath input form

Modify the code for inputting CIMObjectPaths for operations such as getInstance, deleteInstance to allow a path to be input with the same form as the instances in addition to the current form where the input mimics the internal string form for a CIMObjectPath path (classname.p1="xxx",p2="yyy ...). This is simply a convince to bypass the excessive use of quotation marks, commas, etc. required today when CIM object paths are built.  We would keep the current form but also allow the alternate form

The current CLI operation

     cimcli gi CIM_MyClass.key=\"fred\",age=3

could be input as 

      cimcli gi CIM_MyClass key=fred age=3

and cimcli would complete the building of the CIM object path.

The input would allow either format.  Note that this does not conflict with the other option for getInstance, the interactive mode.  If the user provides only the class name the interactive mode is selected.  If key=value pairs are included the NVPN input is used.  If the input includes Class.<keybindings> the CIMObjectPath string form is decoded.

2.5 Add new function testInstance


This function is be based on the same syntax as getInstance.  It does the following:

1. get the class
2. build the instance from input (same as createInstance)
3. build an instance path from this new instance.
4. perform a getinstance based on the instance path created.
5. compare the properties in the created instance to the returned instance.  If the values are different, it will exit cimcli with an error code.  Otherwise it will exit with status zero.  In verbose mode it will display the actual differences.

the following is an example of the use of this in a scripted test scenario in bash.

       TC="Test_CLITestProviderClass"
       NC="-n test/TestProvider"
       MC="$TC -n $NC"

       cimcli ci $MC id=ti1 scalBool=true
       cimcli ti $MC id=ti1 scalBool=false
       if [ "$?" != 0 ]; then
             echo Returned Error code = "$?"
       fi
       cimcli di ${TC}.Id=\"ti1\" ${NC}

This function allows cimcli to be used as a more effective test tool that previous versions in that it can at least do tests on property values against instances in the environment.

NOTE: What we can not do yet is a) compare for existence of properties, b) compare for other comparators for properties (gt, lt, etc.) c) compare returned parameters.

3.0 Work Tasks

This work will include:

  1. Update the code for new functionality
  2. Update documentation (cimcli/doc)
  3. Update tests to test for new functionality and extend current test functionality to increase code coverage in automatic tests.  This will be done by adding tests to the CLITestProvider so that complete tests on the instance manipulation can be accomplished.
  4. Update help documentation in the code.
3. Rationale

Enhance the ability to test with this tool.  Prepare the tool to be released with the next version of Pegasus as part of the public tool set

4. Schedule


Code is ready for review once we have agreed on the PEP. 

Discussion

Issues from v1 to 1.2 were:

1. Modify the syntax so that it more closely matches MOF syntax

Issues from 1.3 to 1.4

[ (r_kumpf) This command is missing a quote.
(k_schopmeyer) 'fred'
¤41]                     cimcli ci myClass name="fred, age= arrayvar=abc,def,ghi

CONCLUSION: Corrected

==============

[ (r_kumpf) The second definition is redundant.
(k_schopmeyer) Correct. changed from one bnf form to another and did not catch that one.
¤49]     featureValueDefinition = <featureNameDefinition> "=" [value_set] | <featureNameDefinition> "="
CONCLUSION: Corrected
=====================

[ ¤52]

[ (r_kumpf) The shell will try to interpret bracket characters. What protection is needed to prevent that?
(k_schopmeyer) Good catch. We are trying to avoid use of things that get caught by shells. Since we diefined this as future, I missed that. Suggestions for a good character that would clearly separate this prefix from the CIM_Name??
¤53]     typeDefinition = "[" <DMTFDataTypeName> "]"

CONCLUSION: None right now but this is future so suggestions welcome

==========================

[ (r_kumpf) I guess this definition is left as an exercise for the reader. Is there a notation to communicate that?
¤58]     featureName =  <validCIMPropertyOrParameterName>

[ ¤59]

[ (r_kumpf) How is arrayValueConstant defined?
¤60]     valueSet = <valueConstant> | <arrayValueConstant>

CONCLUSION:  Comment added.

=================

[ ¤63]
[ (r_kumpf) Is it really necessary to support all these bases? And since when is a boolean an integer?
(k_schopmeyer) Not really but why not. Why is boolean not an integer? integer and decimal are not equal.
¤64]     integerValueConstant = <binaryValueConstant> | <octalValueConstant> | <decimalValueConstant> |    
[ ¤65]                                              <booleanValueConstant> | <hexValueConstant>


Conclusion:  They are defined here because that is what is supported in StringConversion.h. We are trying to
use what exists.

=================

¤86]

[ (r_kumpf) Why is the character-string-literal level of indirection defined?
(k_schopmeyer) I was lazy and copied.
¤87]     stringValueConstant = <character-string-literal>

Conclusion: Fixed

=========================

¤87]     stringValueConstant = <character-string-literal> 
[ (r_kumpf) Are quote characters actually accepted around the value, or are you expecting the shell to strip them off? If quotes are accepted around the value, how is it determined whether they are meant to be part of the value?
(k_schopmeyer) Quotes are optional and are only required for strings containing spaces, etc. that would end up separating the string value. The goal here is to make it easy for the user to type things in and since quote marks must be escaped in at least some shells, we are trying to leave them off as much as possible.
¤88]     character-string-literal ::=[quote]1*( *stringChar  )[quote]

Conclusion: They are stripped unless escaped.

=================

[ (r_kumpf) Any such deviation from the spec will need to be clearly documented.
(k_schopmeyer) I don't see this as deviation from the spec. This is a support tool and allowing it to do things based on the parameters in the spec seems clearly logical. Allowing the keyword not to tell the software to input the current datetime is a widely used convention and at least in some small way useful. 'This will be documented in the help, etc. It was just a convience to make things easier. Ex. In a test case, it may well be easier to let the system pick a time value from a simple keyword than to have to corretly put in the exact string all the time. we are trying here to make this thing useful.
¤101] A CIM DateTime value is input as a string with the CIM formatting for ascii date time representation. Note that we have expanded this to allow input of some keywords such as "now" that would force cimcli to provide thecurrent time rather than a fixed time input.

CONCLUSION: Discuss
==========================

[ (r_kumpf) Regarding the null vs. empty string determination above, won't the quotes be stripped off by the shell? The comment at tag 88 comes into play also, because what if the string value was really meant to be just two quote characters?
¤102] This allows inputting property value information with a minimum of extra characters.  For example, quotation marks are required for string properties that invoke spaces. The quotation mark would be treated as a special character and would require escaping if it was to be included as part of the input string. The namevalue pair separator would be space (the ShellArguementTerminator).  Thus the special characters are:

Conclusion: We need to account for empty string, NULL string and string that contains only quotes.
================

[ (r_kumpf) What does one do if they need to include a comma character in one of the string values in a string array?
(k_schopmeyer) You got me there I think. That would have to be escaped as a special character.
¤106] NOTE: This differs from the standard MOF input in the limitation on the use of spaces and also the lack of the use of the required { and } to initiate and terminate an array value set. We propose to make the use of the { and } optional.   However, not that in at least some shells the { is an expansion character (ex. in bash a={abc,def} expands to a=abc a=def so that we have to be very careful of the use of this character.

Conclusion: We need to account for this case.
====================
[ (r_kumpf) Does this command output any result? It seems like it would be convenient for scripting if the object path for the new instance were written to stdout.
(k_schopmeyer) Good catch. That one will be added.
¤108] The behavior will be that the instance will be created with just the properties  defined as part of the input using the property/value pairs on the input line.  No extra properties will be added.   The typing for properties will be derived by accessing the CIMClass for property type, array vs. non-array characteristics, etc.

Conslusion: added.

=================

[ ¤120]
[ (r_kumpf) The third sentence appears have been copied from EnumerateInstances (or similar) documentation. It needs to be updated for ModifyInstance.
(k_schopmeyer) Agreed.
¤121] The behavior of this function will be similar to the behavior of create instance except that it honors the property list option.   That option determines the properties that are requested and which may be modified.  Any properties not on the property_list are not inserted into the returned instances.

Conclusion: Fixed

======================

[ ¤144]       cimcli gi CIM_MyClass key=fred age=3

[ (r_kumpf) I can see how this might seem convenient, but it seems misleading. GetInstance takes an object path parameter, and you've invented a new way to represent an object path. I don't see what is so cumbersome about specifying an object path in the standard way. If it contains quote characters, just put single quotes around the whole path.
(k_schopmeyer) I am giving you a choice here. Either enter it as a path form or use the same form you would use if you were entering it as the creation of the key properties of an instance. Believe me, if you do this a bunch, it is easier. That does not mean that the user has to use the separate keyword=value form. However, this means that almost everything except for paths in properties or references can be entered in the keyword=value form.
¤145] and cimcli would complete the building of the cimobject path.

Conclusion: Discuss
================

[ ¤148] 2.5 Add new function testInstance


[ (r_kumpf) This is the kind of feature that screams 'test tool' and 'not suitable for deployment to end users'. Couldn't the same purpose be met by using GetProperty operations and comparing the property value with the expected value?
(k_schopmeyer) At least in part it is a test tool. Yes on a single property basis the get property and a compare could be used. However, we are trying to offer a fairly wide set of user options here from what we have seen users do and ask for so this seems logical and fits with the input formatting structures we have set up. We need to sort out the test tool versus production tool aspects here and I don't care if we find a way to restrict certain things for what might be considered production but for the test group this appeared logical and straight forward.
¤149] This function would be based on the same syntax as getInstance.  It would do the following:
[ ¤150]
[ ¤151] 1. get the class

Conclusion:  Discuss

====================
[ ¤160]        NC="-n test/TestProvider"
[ (r_kumpf) I guess the first 'T' is a typo?
(k_schopmeyer) yes
¤161]        MC="T$TC -n $NC"

Conclusion: Corrected




 


Copyright (c) 2006 Hewlett-Packard Development Company, L.P.; IBM Corp.;
EMC Corporation; Symantec Corporation; The Open Group.

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to
deal in the Software without restriction, including without limitation the
rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
sell copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

THE ABOVE COPYRIGHT NOTICE AND THIS PERMISSION NOTICE SHALL BE INCLUDED IN
ALL COPIES OR SUBSTANTIAL PORTIONS OF THE SOFTWARE. THE SOFTWARE IS PROVIDED
"AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT
LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR
PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

 

Template last modified: March 26th 2006 by Martin Kirk
Template version: 1.11