tagfilter resolving and filtering

[rlad78]

This PR aims to add a @check_tagfilter decorator that can be added to any 'get' or 'list' AXL requests. The decorator is designed to do the following:

1. Check to see that all provided tags in tagfilter are valid 'returnedTags' base values
2. Convert tagfilter list/dict of base tag elements into a nested dict of all necessary child tags
3. Filter out all returned objects' attributes that are not tags included in tagfilter (works on lists of objects too)

Below is an example of how this decorator could be implemented:

from ciscoaxl.helpers import check_tagfilter

...

    @check_tagfilter("getPhone")
    def get_phone(
        self,
        name: str,
        tagfilter={
            "name": "",
            "product": "",
            "description": "",
            "protocol": "",
            "locationName": "",
            "callingSearchSpaceName": "",
        },
    ):
        try:
            return self.client.getPhone(name=name, returnedTags=tagfilter)["return"][
                "phone"
            ]
        except Fault as e:
            return e

Some applications where this is beneficial are...

Resolving tags with children
Currently, supplying {'lines': {'line': ''}} to tagfilter will result in much of the returned line information being None since none of the children of the "line" element are supplied. check_tagfilter will look up the schema for the relevant 'returnedTags' element and add all necessary children to 'lines' in tagfilter.

Without check_tagfilter

ucm.get_phone("CUCFSDEMO01", tagfilter={'lines': {'line': ''}})
"""
{
    'name': None,
    'description': None,
    'product': None,
    'model': None,
    'class': None,
    'protocol': None,
...
    'lines': {
        'line': [
            {
                'index': 1,
                'label': None,
                'display': None,
                'dirn': None,
                'ringSetting': None,
                'consecutiveRingSetting': None,
                'ringSettingIdlePickupAlert': None,
                'ringSettingActivePickupAlert': None,
                'displayAscii': None,
                'e164Mask': None,
                'dialPlanWizardId': None,
                'mwlPolicy': None,
...
    'uuid': '{A8222115-163E-FCB0-1495-A4D952679401}'
}
"""

With check_tagfilter ('line' not needed, only parent node)

ucm.get_phone("CUCFSDEMO01", tagfilter={'lines': ''})
"""
{
    'lines': {
        'line': [
            {
                'index': 1,
                'label': 'OMH - 6220',
                'display': 'OMH',
                'dirn': {
                    'pattern': '5550001',
                    'routePartitionName': 'Staging-PT',
                    'uuid': '{5E01E393-1B13-1677-9B98-D73CFF81F6DD}'
                },
                'ringSetting': 'Ring',
                'consecutiveRingSetting': 'Use System Default',
                'ringSettingIdlePickupAlert': None,
                'ringSettingActivePickupAlert': None,
                'displayAscii': 'OMH',
                'e164Mask': None,
                'dialPlanWizardId': None,
                'mwlPolicy': 'Use System Policy',
...
            }
        ],
        'lineIdentifier': None
    },
    'uuid': '{A8222115-163E-FCB0-1495-A4D952679401}'
}
"""

Flexibility of tagfilter parameters
You can now use a list, dict, or tuple to define which tags you would like to use. For example, all three of these calls have the same result:

ucm.get_phone("CUCFSDEMO01", tagfilter={"name": "", "description": "", "lines": ""}) 
ucm.get_phone("CUCFSDEMO01", tagfilter=['name', 'description', 'lines'])
ucm.get_phone("CUCFSDEMO01", tagfilter=('name', 'description', 'lines'))

If you want to see all tags, you can set tagfilter as None, "", {}, or anything else with a false bool value. This ensures backwards compatibility with previous versions of ciscoaxl.

Return object filtering
Instead of returning a whole bunch of None values for items that were not included in the tagfilter, the returned object will have its attributes filtered and remove any non-requested items.

Without check_tagfilter

ucm.get_phone("CUCFSDEMO01", tagfilter={'name': '', 'description': '', 'model': ''})
"""
{
    'name': 'CUCFSDEMO01',
    'description': 'Jabber-PC/Mac-demo01',
    'product': None,
    'model': 'Cisco Unified Client Services Framework',
    'class': None,
    'protocol': None,
    'protocolSide': None,
    'callingSearchSpaceName': None,
    'devicePoolName': None,
    'commonDeviceConfigName': None,
    'commonPhoneConfigName': None,
    'networkLocation': None,
    'locationName': None,
    'mediaResourceListName': None,
    'networkHoldMohAudioSourceId': None,
    'userHoldMohAudioSourceId': None,
    'automatedAlternateRoutingCssName': None,
    'aarNeighborhoodName': None,
    'loadInformation': None,
    'vendorConfig': None,
...
    'elinGroup': None,
    'ctiid': 15864,
    'uuid': '{A8222115-163E-FCB0-1495-A4D952679401}'
}
"""

With check_tagfilter

ucm.get_phone("CUCFSDEMO01", tagfilter=['name', 'description', 'model'])
"""
{
    'name': 'CUCFSDEMO01',
    'description': 'Jabber-PC/Mac-demo01',
    'model': 'Cisco Unified Client Services Framework',
    'uuid': '{A8222115-163E-FCB0-1495-A4D952679401}'
}
"""

Lifting nested _value_1 results
Some returned items will contain a dict-like object that houses the actual value of the item (_value_1) and the UUID of the item. However, nearly all UCM entities returned as a child of returnedTags don't have any place in the AXL schema where their value can be retrieved via their UUID, making this data-point very unhelpful when compared to the annoyance of having to go down another level to reach the desired data.

check_tagfilter will automatically lift any _value_1 items and assign that value directly to the parent.

Without check_tagfilter

ucm.get_phone("CUCFSDEMO01", tagfilter={'callingSearchSpaceName': ''})
"""
{
    'name': None,
    'description': None,
    'product': None,
    'model': None,
    'class': None,
    'protocol': None,
    'protocolSide': None,
    'callingSearchSpaceName': {
        '_value_1': '864-Clemson-Device-CSS',
        'uuid': '{61A8B6B2-887F-0FC3-71EE-92150BCAB641}'
    },
    'devicePoolName': None,
    'commonDeviceConfigName': None,
    'commonPhoneConfigName': None,
...
    'elinGroup': None,
    'ctiid': 15864,
    'uuid': '{A8222115-163E-FCB0-1495-A4D952679401}'
}
"""

With check_tagfilter

ucm.get_phone("CUCFSDEMO01", tagfilter={'callingSearchSpaceName': ''})
"""
{
    'callingSearchSpaceName': '864-Clemson-Device-CSS',
    'uuid': '{A8222115-163E-FCB0-1495-A4D952679401}'
}
"""

However, this feature would not be backwards compatible with previous version of ciscoaxl. Therefore, it is left disabled by a variable in config.py:

DISABLE_VALUE1_RESOLVER = True


def enable_value1_resolver() -> None:
    global DISABLE_VALUE1_RESOLVER
    DISABLE_VALUE1_RESOLVER = False

To enable the feature, the user must run enable_value1_resolver() at some point in their script. This could be changed in the future, but right not it is necessary in order not to break anything.

---

That's about it! The decorator has not been applied to any methods in axl.py in this PR since that will involve some refactoring those method's arguments (most 'get' methods don't have tagfilter). That would be best for another PR down the road.

wsdl.py also paves the way for automatic doc generation. I tried to comment everything in there as best as I could, but please ask me questions if anything is confusing or not explained well enough.

[rlad78]

Converting back to a draft for now, need to set up my test env so that I can resolve issues in Python 3.7.

[bradh11]

This is a large PR and may take a bit to fully review. That said, it seems a lot of this code sets us up for future improvements and does not modify current behaviors unless you go out of your way to do so (ie will not break existing usage). Is that fair to say?

[rlad78]

This is a large PR and may take a bit to fully review. That said, it seems a lot of this code sets us up for future improvements and does not modify current behaviors unless you go out of your way to do so (ie will not break existing usage). Is that fair to say?

Absolutely. The only modification made to axl.py was this addition in the class' __init__:

self._zeep = axl_client

No other includes have been added to axl.py either, so nothing additional will be run or loaded. All other modules added by this PR are currently unused unless specifically imported at runtime.

And I apologize for the size of the PR. Most of the functionality is carried over from my personal fork where it was first implemented, but now modified and tested to fit the needs of this project. The bulk of it resides in wsdl.py, which is a helper library for dealing with WSDL elements.Since all 3 main enhancements of this PR use that library in some way, it was difficult to try and split it up.

If there's any way I can assist, whether that be making changes, answering questions, or developing tests for this PR, please don't hesitate to ask.

[bradh11]

It makes it easier since it’s not really touching the main class object. I believe the wsdl helper methods will be quite useful and we might want to bring it under the __init__ method of the main class at some point. I think we should be able to merge this fairly soon.

[rlad78]

There may actually be a couple things I could remove from wsdl.py that aren't currently used by anything in this PR. I'll try and look into what I can leave out sometime tomorrow. It can always be re-added if it's needed.

[bradh11]

There may actually be a couple things I could remove from wsdl.py that aren't currently used by anything in this PR. I'll try and look into what I can leave out sometime tomorrow. It can always be re-added if it's needed.

Don’t pull anything out just yet. I think this has the helper library you showed Jeff and I that maps out the wsdl data structure. That’s really useful. If there are other helper functions that are not relevant - call them out in comments here for discussion and we can decide together whether they stay or go (if any at all).

[rlad78]

In wsdl.py:

Line 548 print_element_layout()
Line 562 print_required_element_layout()
Line 575 print_return_tags_layout()

These are dev tools for visualizing element structures, but aren't necessarily needed for the enhancements this PR is trying to bring. They are all basically convenience functions that wrap around the main class AXLElement's print_tree method.

Line 595 validate_arguments()

This is a useful function for validating user kwargs in 'add' and 'update' AXL requests. Uses AXLElement.validate() which will raise helpful exceptions to indicate to the user what mistake they have made. Again, not relevant to the current PR. It could be re-implemented in the future, but I liked your idea of using Pydantic better and think that would be more intuitive and easier to maintain.