Was messing around with my VMware PKS and NSX-T nested environment today and in the process broke it. Broke it beyond the point PKS couldn’t cleanup NSX-T, don’t ask!! See here how to delete a cluster. If interested in building out your own PKS on NSX-T environment, see my series PKS NSX-T Home Lab series here.
Objects created in NSX-T by PKS are created by a Superuser we defined during the PKS install process. These objects are known as protected objects. Protected objects created by a superuser cannot be deleted by an admin. In NSX Manager you can see what is a protected object by the icon shown below. It can also be seen that the Delete button is greyed out.
To delete protected objects we need to use the API. NSX-T API documentation can be found here. But in an effort to save you trawling threw the documentation, below are example calls to delete all the objects created when creating a Kubernetes cluster. All calls have the header of “X-Allow-Overwrite: true” which is used so we can delete the protected objects. Of course replace the credentials with your own and the ID’s of the objects.
Delete Logical Switch
curl -k -u admin:VMware1! -X DELETE ‘https://nsx-mgr.lab.keithlee.ie/api/v1/logical-switches/12f3ef66-760e-4231-bef8-d3fe129898fc?detach=true&cascade=true’ -H “X-Allow-Overwrite: true”
Delete Logical Router
curl -k -u admin:VMware1! -X DELETE ‘https://nsx-mgr.lab.keithlee.ie/api/v1/logical-routers/75ff754d-db47-4465-a880-1c06ec86b687?force=true’ -H “X-Allow-Overwrite: true”
Delete Logical Router Router Ports
curl -k -u admin:VMware1! -X DELETE ‘https://nsx-mgr.lab.keithlee.ie/api/v1/logical-router-ports/0806b919-201a-467b-8ccc-820a02b4ca31?force=true’ -H “X-Allow-Overwrite: true”
Delete NAT Rules
curl -k -u admin:VMware1! -X DELETE ‘https://nsx-mgr.lab.keithlee.ie/api/v1/logical-routers/6585c274-b395-4189-a7b6-c30dc6c38d4d/nat/rules/2064’ -H “X-Allow-Overwrite: true”
Delete IPAM Subnet
curl -k -u admin:VMware1! -X DELETE ‘https://nsx-mgr.lab.keithlee.ie/api/v1/pools/ip-subnets/5ba541d9-7220-4d8d-8683-3cc28f4566a7’ -H “X-Allow-Overwrite: true”
Delete IP Pools
curl -k -u admin:VMware1! -X DELETE ‘https://nsx-mgr.lab.keithlee.ie/api/v1/pools/ip-pools/4e6bfe2a-a4f9-4701-9756-79608ad10d61?force=true’ -H “X-Allow-Overwrite: true”
Release IP from IP Pool
curl -k -u admin:VMware1! -X POST ‘https://nsx-mgr.lab.keithlee.ie/api/v1/pools/ip-pools/d11c1c51-ff8f-45cd-b4ab-bed93cf8a02d?action=RELEASE’ -H “X-Allow-Overwrite: true” -d ‘{“allocation_id”:”10.0.80.11″}’ -H “Content-Type: application/json”
Delete Load Balancer
curl -k -u admin:VMware1! -X DELETE ‘https://nsx-mgr.lab.keithlee.ie/api/v1/loadbalancer/services/bb352eaf-0764-46a3-9880-4a382bb8539a’ -H “X-Allow-Overwrite: true”
Delete Virtual Servers
curl -k -u admin:VMware1! -X DELETE ‘https://nsx-mgr.lab.keithlee.ie/api/v1/loadbalancer/virtual-servers/0cdb061f-6003-4d90-9895-f5bfed3ab508?delete_associated_rules=true’ -H “X-Allow-Overwrite: true”
Delete Load Balancer Pools
curl -k -u admin:VMware1! -X DELETE ‘https://nsx-mgr.lab.keithlee.ie/api/v1/loadbalancer/pools/1ce234c8-d3fc-4d9a-a2d9-e8c29c20c9cf’ -H “X-Allow-Overwrite: true”
Delete Load Balancer Monitor
curl -k -u admin:VMware1! -X DELETE ‘https://nsx-mgr.lab.keithlee.ie/api/v1/loadbalancer/monitors/d3baf9b7-8308-4b6e-9342-e2b1f8e81d1e’ -H “X-Allow-Overwrite: true”
Delete Group
curl -k -u admin:VMware1! -X DELETE ‘https://nsx-mgr.lab.keithlee.ie/api/v1/ns-groups/e80abffa-26f6-4de9-a419-0fb0505ef316?force=true’ -H “X-Allow-Overwrite: true”
Thanks for this; I’m glad the process has improved since https://community.pivotal.io/s/article/How-to-delete-orphan-NSX-T-objects-protected-by-superuser where you had to recreate the superuser. THAT was a pain.
My contribution to this post is that I used in this format:
Delete Load Balance
curl -k -X DELETE “https://${NSX_MANAGER_IP}/api/v1/loadbalancer/services/4c90ca51-5249-48cf-aaf9-d431833c0039” -u “$NSX_MANAGER_USERNAME:$NSX_MANAGER_PASSWORD” -H “X-Allow-Overwrite: true”
using exports.
The other one is that those API calls make easy the deletion of objects I wnet thru the link as well:
https://community.pivotal.io/s/article/How-to-delete-orphan-NSX-T-objects-protected-by-superuser?t=1547584711477
and for more details about the gross of doing in that way is that you have to remove the principal identity so in my case I got confused since I got one from PKS and one as pks-nsxt-superuser, so I leave it and then replace the method using the API calls on this post.
Thanks you for sharing