Release 4.2(4i) became available on April 23, 2020.
New software features
Enhancements for remote leaf switches
Starting with release 4.2(4), the following enhancements have been introduced for remote leaf switches:
- Support for 10 Mbps as a minimum bandwidth in the IPN
- Support to create an 802.1Q tunnel between the remote leaf switch and the ACI main datacenter
For more information, see the chapter "Remote Leaf Switches" in the Cisco APIC Layer 3 Networking Configuration Guide, Release 4.2(x).
Guidelines and restrictions:
- The IPN path is only used for managing remote leaf switches (management functions such as downgrades, discovery, COOP, and policy pushes).
- All traffic from the Cisco ACI datacenter and remote leaf switches is through the local L3Out.
- The EPG or bridge domain are not stretched between the remote leaf switch and the ACI main datacenter.
IGMP snooping version 2 group scale increase
IGMP snooping now supports 32k groups.
Layer 3 multicast VRF scale increase
Layer 3 multicast now supports 300 VRF tables fabric-wide.
Multipod leaf switch scale increase
A multipod environment now supports up to 400 leaf switches per pod.
Network Insights-Base app...
... is now prepackaged with Cisco APIC
Support for 25 SCVMM domains with 10k endpoints
A Cisco ACI fabric can now have up to 25 SCVMM domains with 10k endpoints in pre-provisioned mode.
Support for custom EPG names for VMM domains
You can give EPGs a custom name that carries over to a VMware vCenter port group or a Microsoft VM network. The feature is available for VMware vSphere Distributed Switch, Microsoft System Center Virtual Machine Manager (SCVMM), and Cisco ACI Virtual Edge. If you do not provide a custom name, the domain association assigns one.
For more information, see the "Custom EPG Names and Cisco ACI" chapter in the Cisco ACI Virtualization Guide, Release 4.2(x).
Guidelines and restrictions:
Giving an EPG a custom name — a beta preview feature in the Cisco APIC 4.2(3) release — is in general availability in this release.
User lockout after continuous failed attempts to login
You can block a user from being able to log in after the user fails a configured number of login attempts. You can specify how many failed login attempts the user can have within a specific time period. If the user fails to log in too many times, then that user becomes unable to log in for a specified period of time.
For more information, see the section "User Lockout After Continuous Failed Attempts to Log in" in the chapter "Access, Authentication, and Accounting" in the Cisco APIC Security Configuration Guide, Release 4.2(x).
This is an enhancement to allow for text-based banners for the Cisco APIC GUI login screen.
For a client (browser or ssh client) that is using IPv6, the Cisco APIC aaaSessionLR audit log shows "0.0.0.0" or some bogus value.
Description fields are not available for resource pools (VLAN, VSAN, Mcast, VXLAN etc).
This is an enhancement request for allowing DVS MTU to be configured from a VMM domain policy and be independent of fabricMTU.
A leaf switch port flaps without raising a warning.
When trying to track an AVE endpoint IP address, running the
show endpoint ip x.x.x.x command in the Cisco APIC CLI to see the IP address and checking the IP address on the EP endpoint in the GUI shows incorrect or multiple vPC names.
The API query
/api/class/compCtrlr.json?rsp-subtree=full? returns a malformed JSON file.
When a user logs into the Cisco APIC GUI and selects the SAL login domain, the authorization fails and the user gets thrown back to the initial login screen. The Cisco APIC NGINX logs show a failure to parse the AVPair value that is sent back by the SAML IDP. When checking the AVPair value returned by the Okta SAML IDP
<inRole value="shell:domains=all//read-all"/>, the value seems to have correct syntax.
There is a minor memory leak in
svc_ifc_policydist when performing various tenant configuration removals and additions.
Cisco ACI UCSM integration does not work as expected. The Cisco APIC cannot discover a loose node UCS Fabric interconnect 6400 series when it is connected to the Cisco ACI fabric with a 100G interface.
Dynamic VLANs are programmed on interfaces that are not associated to the VLAN pool/AEP. This behaviour is seen when a UCS Fabric Interconnect blade switch has multiple uplinks to the fabric. Although some of those uplinks are mapped to a different AEP and the is EPG set for pre-provision, dynamic EPGs still are reported for that EPG.
After removing and re-applying the IP SLA monitoring policy on a PBR policy, tracking does not work correctly.
Upgrading to the 4.2(1i) release, Layer 3 packet drops are no longer seen, but Layer 3 drop flows are still seen. However, Layer 3 drop flows do not give as much information.
An endpoint is unreachable from the leaf node because the static pervasive route (toward the remote bridge domain subnet) is missing.
A native VLAN for a VMM domain does not work if resolution immediacy is set to pre-provision. In this case, the untag policy is pushed to VMware vCenter and a port group is created (this is expected). However, the policy is programed as trunk on switch side, which prevents the ESXi vmkernal and switch from communicating.
Randomly, the Cisco APIC GUI alert list shows an incorrect license expiry time.Sometimes it is correct, while at others times it is incorrect.
If pre-provision is not in place, there can be a complete outage to VMM integrated endpoints. If the host discovery is not successful, the policy will not be dynamically pushed to the leaf switches because virtual machines are attached.
RADIUS authentication cannot be configured from the Cisco APIC GUI.
An SNMP v3 trap is sent 2 minutes after a PSU is removed from the Cisco APIC, and a core file for the eventmgr is generated.
For a DVS with a controller, if another controller is created in that DVS using the same host name, the following fault gets generated: "hostname or IP address conflicts same controller creating controller with same name DVS".
The Cisco APIC GUI hangs on a loading screen when trying to configure interfaces policies from the following location: Fabric -> Inventory -> Pod -> Leaf switch -> Interface tab -> Configuration mode.
A Cisco ACI Virtual Edge host configured with Protective HA on the cluster might not come out of Quarantine mode.
In a Fabric Interconnect topology, a vPC may not be detected by the OpflexAgent on a HyperV host.
App techsupport collection does not work sometimes when triggered from the Cisco APIC GUI.
In Cisco ACI Virtual Edge, there are faults related to VMNICs. On the Cisco ACI Virtual Edge domain, there are faults related to the HpNic, such as
Fault F2843 reported for AVE | Uplink portgroup marked as invalid.
Host subnets (/32) that are created under an SCVMM-integrated EPG get pushed as a virtual machine subnet under the virtual machine network in SCVMM. Virtual machine networks on SCVMM do not support /32 virtual machine subnets and fail to come up. Virtual machines that were previously associated to the virtual machine networks lose connectivity.
A leaf switch crashes with the following reason:
Reason: reset-triggered-due-to-ha-policy-of-reset Service: vleaf_elem hap reset
Configuration rollback fails with the following error:
VRF Validation failed for VRF = : - ARP policy default in uni/tn-Prod/out-PROD_L3OUT/Inodep - L3OUT_PROD_LEAF103/lifp-PROD_L3OUT_INTERFACE/rsArplfPol is currently not supported on the interface
An admin read-only user can not see the System Settings tab in the Cisco APIC GUI.
The plgnhandler process crashes on the Cisco APIC, which causes the cluster to enter a data layer partially diverged state.
When physical domains and external routed domains are attached to a security domain, these domains are mapped as associated tenants instead of associated objects under Admin > AAA > security management > Security domains.
Special characters are not allowed in the GUI for the SNMP community string, but you can still post a configuration that has special characters in the string by using the REST API.
In Cisco APIC release 4.2, the opflex_agent continuously disconnects from the leaf switch approximately every 5 minutes. This results in cores being generated for
vleaf_elem processes on the switch.
When the PSU is powered off, a fault indicates that it is in a failed state.
In the Cisco APIC GUI, go to Admin->Firmware->Infrastructure->Nodes. Open an existing update group. While the group loads, the following text appears:
Click on + to add nodes to Node Upgrade Group. The text disappears after the nodes are loaded. The update groups cannot be edited (there is no "+" or "trash" symbol).
OpenStack supports more named IP protocols for service graph rules than are supported in the Cisco APIC OpenStack Plug-in.
A Cisco APIC-generated CSR contains the
unstructuredName field, which does not work with some CA certificates.
When togging between "Configured and Operational" under Tenants >Tenant_name > Contracts > Contract_name > Topology, contract lines are not visible when the toggle is on operational mode even though contracts are still operational.
Clicking on Fabric --> Access Policies --> Interfaces --> Leaf Interfaces --> Profiles --> <any_profile> --> "Show Usage" --> "Nodes using this policy" --> "Usage details of node" results in logging off the user and freezing the GUI screen.
A Cisco ACI leaf switch does not have MP-BGP route reflector peers in the output of
show bgp session vrf overlay-1. As a result, the switch is not able to install dynamic routes that are normally advertised by MP-BGP route reflectors. However, the spine switch route reflectors are configured in the affected leaf switch's pod, and pod policies have been correctly defined to deploy the route reflectors to the leaf switch. Additionally, the bgpPeer managed objects are missing from the leaf switch's local MIT.
In a GOLF configuration, when an L3Out is deleted, the bridge domains stop getting advertised to the GOLF router even though another L3Out is still active.
Name column of the the output of the
show zoning-rule CLI command that is executed on a leaf switch running a 14.x release does not populate all of the expected contracts names. This issue makes it difficult to identify which rule ID is associated to which contract from the
show zoning-rule command that is executed on a given leaf switch.
The CLI command
show interface x/x switchport shows VLANs configured and allowed through a port. However, when going to the GUI under Fabric > Inventory > node_name > Interfaces > Physical Interfaces > Interface x/x > VLANs, the VLANs do not show.
When viewing leaf switch interface profiles in access policies, the list cannot be sorted by name or description.
The following fault is raised on a Cisco ACI fabric that has VMM/UCS integration:
F609530 ([FSM:FAILED]: Send configuration update to External Device Manager Regarding the Dom Def(TASK:ifc:policymgr:ExtdevRsDomDefConfigDomDef).
Immediately after a Cisco APIC cluster upgrade, all EPG SCVMM networks are marked for deletion. Networks not attached to virtual machines are deleted. Networks that are attached to virtual machines fail to get deleted, as they are being used and the following fault gets raised on the Cisco APIC cluster for each network:
F1471EPG deployment failed due to Powershell call failed. Error Message: Cannot Delete VmNetwork
Time zone/local time on a Cisco APIC and switches differ when set to the EET timezone.
Selecting the RADIUS login domain from the GUI results in the following error:
Error: 400 - unknown property value test, name realm, class aaaConsoleAuth [(Dn0)] Dn0=uni/userext/authrealm/consoleauth,
tmpfs file system that is mounted on
/data/log becomes 100% utilized.
The Ciphers drop-down list is not visible in a management access policy.
The SSL Cipher Configuration table is too small. Second row is cut off even when scrolling to the bottom of the table.
It is difficult to configure interface selectors in the GUI, because "interface policy group" window is too narrow.
It is difficult to configure interface selectors, because there is no search option available for the interface policy group window.
The Cisco APIC PSU voltage and amperage values are zero.
SNMP does not respond to GETs or sending traps on one or more Cisco APICs despite previously working properly.
When navigating to System -> Controllers -> Cluster as Seen by Node for any Cisco APIC, the following error displays:
The Request failed due to a server-side error.
Fault F3243 will be raised when changing the VMM configuration if the VMM domain has already been associated to the EPG, even though the change is not related to the current configuration.
The L3Out wizard shows the incorrect
router_id from another VRF table.
Service Graph rendering fails if a service graph is attached to a unidirectional filter in a contract subject. For example:
- filter chain for provider to consumer: use service graph with PBR
- filter chain for consumer to provider: no service graph
126.96.36.199.188.8.131.52.184.108.40.206.2 in v1 SNMP trap
cefcPowerStatusChange by Cisco APIC is observed.
Cisco APIC apps do not have connectivity using an inband network.
If a FEX hardware model is N2K-C2348UPQ-10GE, this FEX does not consume a FEX_48_10G license.
You might not be able to log in to a Cisco ACI leaf or spine switch.
TACACS external logging is not supported at the tenant level.
Hosts that require a DHCP-obtained address (Hyper-V, for example) from the Cisco APIC do not work. Checking the DHCP logs shows the DHCP discovers coming in frequently.
The policymgr DME process can crash because of an OOM issue, and there are many
pcons.DelRef managed objects in the DB.
A Cisco APIC might report high memory utilization when polling through SNMP.
Zookeeper creates transactions files when the cluster is converging. During long periods of network unreachability, these files may get created at a more frequent rate, leading to space filling up.
The following symptoms are present:
- The event manager generates a core
- The APIC cluster is in a diverged state
- The event manager is not running on APIC 1 and 2
- Service 3 shards are impacted
After upgrading the leaf switch, the "Allowed VLANs" list in the UCSM integration is no longer updated with the current list of VLANs deployed along the associated paths.
After removing a configuration from Cisco ACI Multi-Site, the fabric nodes started reloading.
show system reset-reason command shows the following:
Reason: reset-triggered-due-to-ha-policy-of-reset Service:policy_mgr hap reset
In a transit L3Out, after adding one new static route (a subnet of a summary route) on a border leaf switch, the OSPF summary route disappears from the route table of the border leaf switch because the route is deleted.
This is an enhancement to include the managed object class name and
isPersisted attribute in the DME log line.
VPC protection created in prior to the 2.2(2e) release may not to recover the original virtual IP address after fabric ID recovery. Instead, some of vPC groups get a new vIP allocated, which does not get pushed to the leaf switch. The impact to the dataplane does not come until the leaf switch had a clean reboot/upgrade, because the rebooted leaf switch gets a new virtual IP that is not matched with a vPC peer. As a result, both sides bring down the virtual port channels, then the hosts behind the vPC become unreachable.
In a Cisco vAPIC environment in which the administrative state of Eth1/2 is down, fault F0106 is presented for Eth1/2 of the Cisco vAPIC nodes.
Updating the interface policy group breaks LACP if eLACP is enabled on a VMM domain. If eLACP was enabled on the domain, Creating, updating, or removing an interface policy group with the VMM AEP deletes the basic LACP that is used by the domain.
When shuting down a leaf switch interface that is connected a Cisco APIC node, even if the Cisco APIC interface shows as down, the status in the GUI is not changed. You can view the interface status by going to:
Admin > Controllers > APIC > controller-APIC
Fault F1527 is raised when the
/data/log directory is over 75% full. The
/data/log directory contains a large amount of gzipped 21M
svc_ifc_licensemgr.bin.warnplus.log files. The
/data/log directory does not reach 80% or 90% full.
Fault F0135 is raised when using an AVE VMM domain, stating:
Unsupported remote operation detected on EPG: detected in controller: controller-ip with name controller-name in datacenter dc-name in domain vmm-domain-name, error [VLAN is set to none for port group on vcenter but untagged access is not enabled for EPG]
A switch entered into a bootloop and an upgrade is triggered multiple times if the maintenance policy is pushed with a REST API call that has the incorrect version.
The VMM endpoint data plane verification function does not work well when a blade switch is in the middle. This might cause an unexpected DVS detach, or the VMM EPG VLAN might be removed on the leaf switch interface.
When migrating an EPG from one VRF table to a new VRF table, and the EPG keeps the contract relation with other EPGs in the original VRF table. Some bridge domain subnets in the original VRF table get leaked to the new VRF table due to the contract relation, even though the contract does not have the global scope and the bridge domain subnet is not configured as shared between VRF tables. The leaked static route is not deleted even if the contract relation is removed.
Packet loss is observed across the spine switches for unicast traffic.
For Cisco APIC, snmpwalk/get returns unexpected values for object
The CSSM receives an extremely high number of entitlement requests from a Cisco ACI fabric, in upwards of 25,000 over a 48-hour period.
A stale prefix entry remains that is associated with an old VRF table.
A VMMmgr crash maybe observed in a scaled environment with 20+ Floating L3Outs.
When using the Internet Explore browser, there is console error. This error will break some pages under Fabric -> Inventory -> [ANY POD] -> [ANY LEAF] / [ANY SPINE] -> Interfaces -> Physical, PC, VPC, FC, FC PC.
Legacy mode bridge domain is intended for a specific use case that requires higher bridge domain (VLAN) numbers per switch. As of Cisco APIC release 4.2, ~2000 normal bridge domains can be deployed on the same leaf switch, while 3500 legacy mode bridge domains can be deployed on the same leaf switch. However, as a trade-off for the bridge domain (VLAN) numbers, legacy mode bridge domains lose various Cisco ACI-specific capabilities, such as contracts, pervasive gateway (bridge domain subnet). This is an enhancement to update the name of Legacy Mode for a bridge domain to reflect its functionality and purpose in the Cisco APIC GUI. With this enhancement, Legacy Mode is presented as Scaled L2 Only Mode.
A Cisco vAPIC running release 4.2(3l) goes into a
sh-4.2# prompt. Runing any command returns
Admin cookie not found.
No new flows are configured in OVS, and the OpFlex agent log shows the following message:
[error] [ovs/PolicyStatsManager.cpp:461:sendRequest] Failed to send policy statistics request: Transport endpoint is not connected
A Cisco ACI Virtual Edge EPG is not programmed on a port channel toward the blade switch after it is deleted and recreated.
When using Cisco APIC release 4.2(2l), the UCSM integration icon is broken in the GUI.
show run leaf|spine <nodeId> command might produce an error for scaled up configurations.
With a uniform distribution of EPs and traffic flows, a fabric module in slot 25 sometimes reports far less than 50% of the traffic compared to the traffic on fabric modules in non-FM25 slots.
Switch upgrades fail with the following error:
Version not compatible.
When you click Restart for the Microsoft System Center Virtual Machine Manager (SCVMM) agent on a scaled-out setup, the service may stop. You can restart the agent by clicking Start.
One of the following symptoms occurs:
- App installation/enable/disable takes a long time and does not complete.
- Nomad leadership is lost. The output of the
acidiagscheduler logs members command contains the following error:
Error querying node status: Unexpected response code: 500 (rpc error: No cluster leader)
The CRC and stomped CRC error values do not match when seen from the APIC CLI compared to the APIC GUI. This is expected behavior. The GUI values are from the history data, whereas the CLI values are from the current data.
Upgrading Cisco APIC from a 3.x release to a 4.x release causes Smart Licensing to lose its registration. Registering Smart Licensing again will clear the fault.
Issues without bug ID
- If you use the REST API to upgrade an app, you must create a new
firmware.OSourceto be able to download a new app image.
- In a multipod configuration, before you make any changes to a spine switch, ensure that there is at least one operationally "up" external link that is participating in the multipod topology. Failure to do so could bring down the multipod connectivity. For more information about multipod, see the Cisco Application Centric Infrastructure Fundamentals document and the Cisco APIC Getting Started Guide.
- With a non-english SCVMM 2012 R2 or SCVMM 2016 setup and where the virtual machine names are specified in non-english characters, if the host is removed and re-added to the host group, the GUID for all the virtual machines under that host changes. Therefore, if a user has created a micro segmentation endpoint group using "VM name" attribute specifying the GUID of respective virtual machine, then that micro segmentation endpoint group will not work if the host (hosting the virtual machines) is removed and re-added to the host group, as the GUID for all the virtual machines would have changed. This does not happen if the virtual name has name specified in all english characters.
- A query of a configurable policy that does not have a subscription goes to the policy distributor. However, a query of a configurable policy that has a subscription goes to the policy manager. As a result, if the policy propagation from the policy distributor to the policy manager takes a prolonged amount of time, then in such cases the query with the subscription might not return the policy simply because it has not reached policy manager yet.
- When there are silent hosts across sites, ARP glean messages might not be forwarded to remote sites if a 1st generation ToR switch (switch models without -EX or -FX in the name) happens to be in the transit path and the VRF is deployed on that ToR switch, the switch does not forward the ARP glean packet back into the fabric to reach the remote site. This issue is specific to 1st generation transit ToR switches and does not affect 2nd generation ToR switches (switch models with -EX or -FX in the name). This issue breaks the capability of discovering silent hosts.
CIMC Version recommendations
- 4.1(1d) CIMC HUU ISO (recommended) for UCS C220 M5 (APIC-L3/M3)
- 4.1(1c) CIMC HUU ISO (recommended) for UCS C220 M4 (APIC-L2/M2)
- 4.0(4e) CIMC HUU ISO for UCS C220 M5 (APIC-L3/M3)
- 4.0(2g) CIMC HUU ISO (recommended) for UCS C240 M4 and M5 (APIC-L2/M2 and APIC-L3/M3)
- 4.0(2g) CIMC HUU ISO for UCS C220 M4 and M5 (APIC-L2/M2 and APIC-L3/M3)
- 4.0(1a) CIMC HUU ISO for UCS C220 M5 (APIC-L3/M3)
- 3.0(4l) CIMC HUU ISO (recommended) for UCS C220/C240 M3 (APIC-L1/M1)
- 3.0(4d) CIMC HUU ISO for UCS C220/C240 M3 and M4 (APIC-L1/M1 and APIC-L2/M2)
- 3.0(3f) CIMC HUU ISO for UCS C220/C240 M4 (APIC-L2/M2)
- 3.0(3e) CIMC HUU ISO for UCS C220/C240 M3 (APIC-L1/M1)
- 2.0(13i) CIMC HUU ISO
- 2.0(9c) CIMC HUU ISO
- 2.0(3i) CIMC HUU ISO