This is the first part of a blog series where I will demonstrate how to create a new VM Cluster in an Exadata Cloud@Clustomer environment using the OCI command-line interface. In the first part, I will create the required VM Cluster Network.
Table of Contents
Create JSON Template
The easiest way to create a new VM Cluster Network is using a JSON template instead of providing all required parameters directly. With the help of the following command, a JSON template can be created.
$> oci db vm-cluster-network create --generate-full-command-json-input
{
"compartmentId": "string",
"definedTags": {
"tagNamespace1": {
"tagKey1": "tagValue1",
"tagKey2": "tagValue2"
},
"tagNamespace2": {
"tagKey1": "tagValue1",
"tagKey2": "tagValue2"
}
},
"displayName": "string",
"dns": [
"string",
"string"
],
"exadataInfrastructureId": "string",
"freeformTags": {
"tagKey1": "tagValue1",
"tagKey2": "tagValue2"
},
"maxWaitSeconds": 0,
"ntp": [
"string",
"string"
],
"scans": [
{
"hostname": "string",
"ips": [
"string",
"string"
],
"port": 0,
"scanListenerPortTcp": 0,
"scanListenerPortTcpSsl": 0
},
{
"hostname": "string",
"ips": [
"string",
"string"
],
"port": 0,
"scanListenerPortTcp": 0,
"scanListenerPortTcpSsl": 0
}
],
"vmNetworks": [
{
"domainName": "string",
"gateway": "string",
"netmask": "string",
"networkType": "string",
"nodes": [
{
"hostname": "string",
"ip": "string",
"vip": "string",
"vipHostname": "string"
},
{
"hostname": "string",
"ip": "string",
"vip": "string",
"vipHostname": "string"
}
],
"vlanId": "string"
},
{
"domainName": "string",
"gateway": "string",
"netmask": "string",
"networkType": "string",
"nodes": [
{
"hostname": "string",
"ip": "string",
"vip": "string",
"vipHostname": "string"
},
{
"hostname": "string",
"ip": "string",
"vip": "string",
"vipHostname": "string"
}
],
"vlanId": "string"
}
],
"waitForState": [
"CREATING|REQUIRES_VALIDATION|VALIDATING|VALIDATED|VALIDATION_FAILED|UPDATING|ALLOCATED|TERMINATING|TERMINATED|FAILED"
],
"waitIntervalSeconds": 0
}
Now it is time to provide all required information. Here is an example of a VM Cluster Network for an ExaCC machine with two compute nodes (quarter rack).
{
"compartmentId": "ocid1.compartment.oc1..xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
"definedTags": {},
"displayName": "clu1net",
"dns": [
"10.123.123.20",
"10.123.123.21"
],
"exadataInfrastructureId": "ocid1.exadatainfrastructure.oc1.eu-frankfurt-1.xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
"freeformTags": {},
"maxWaitSeconds": 0,
"ntp": [
"10.123.123.1"
],
"scans": [
{
"hostname": "myclusterscan",
"ips": [
"10.123.123.15",
"10.123.123.16",
"10.123.123.17"
],
"port": 1521,
"scanListenerPortTcp": 1521,
"scanListenerPortTcpSsl": null
}
],
"vmNetworks": [
{
"domainName": "domain.world",
"gateway": "192.168.0.1",
"netmask": "255.255.255.192",
"networkType": "BACKUP",
"nodes": [
{
"hostname": "node1bak",
"ip": "192.168.0.2",
"vip": null,
"vipHostname": null
},
{
"hostname": "node2bak",
"ip": "192.168.0.3",
"vip": null,
"vipHostname": null
}
],
"vlanId": "1234"
},
{
"domainName": "domain.world",
"gateway": "10.123.123.1",
"netmask": "255.255.255.128",
"networkType": "CLIENT",
"nodes": [
{
"hostname": "node1",
"ip": "10.123.123.18",
"vip": "10.123.123.19",
"vipHostname": "node1vip"
},
{
"hostname": "node2",
"ip": "10.123.123.20",
"vip": "10.123.123.21",
"vipHostname": "node2vip"
}
],
"vlanId": "4321"
}
],
"waitForState": [
"CREATING"
],
"waitIntervalSeconds": 0
}
Node Subsetting
If you want to use the Node Subsetting feature to create smaller clusters (fewer nodes than available compute nodes) to save resources and IP addresses you might run into the following error.
ServiceError:
{
"code": "InvalidParameter",
"message": "Invalid backup network node details. The required number of nodes is 4.. Please visit https://docs.oracle.com/en-us/iaas/Content/API/References/apierrors.htm to learn more about this error code",
"opc-request-id": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
"status": 400
}
Currently, it is not supported to create a VM Cluster Network with fewer nodes than available compute nodes. Even when you don’t need additional nodes, you have to provide IP addresses for these nodes.
Create VM Cluster Network
Use the following call to create the VM Cluster Network. In this example, I will use my JSON file /tmp/clu1net.json.
$> oci db vm-cluster-network create --from-json "file:///tmp/clu1net.json"
After the VM Cluster Network was created its state is REQUIRES_VALIDATION. In the next step, the configuration needs to be validated. For the validation, the OCID of the new VM Cluster Network is required.
$> oci db vm-cluster-network list \
--compartment-id ocid1.compartment.oc1...xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
--exadata-infrastructure-id ocid1.exadatainfrastructure.oc1.eu-frankfurt-1.xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
--query "data[].{\"Name\" : \"display-name\", \"OCID\" : \"id\"}"
[
{
"Name": "clu1net",
"OCID": "ocid1.vmclusternetwork.oc1.eu-frankfurt-1.xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
}
]
$> oci db vm-cluster-network validate \
--exadata-infrastructure-id ocid1.exadatainfrastructure.oc1.eu-frankfurt-1.xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
--vm-cluster-network-id ocid1.vmclusternetwork.oc1.eu-frankfurt-1.xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Troubleshooting
If the validation fails, you can download the validation report to see more information about the errors.
$> oci db vm-cluster-network download-validation-report \
--exadata-infrastructure-id ocid1.exadatainfrastructure.oc1.eu-frankfurt-1.xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
--vm-cluster-network-id ocid1.vmclusternetwork.oc1.eu-frankfurt-1.xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
--file clu1net_validation_report.json
This command will fail when the VM Cluster Network is already in the state VALIDATED.
ServiceError:
{
"code": "IncorrectState",
"message": "The operation cannot be performed because the Exadata Networks with ID ocid1.vmclusternetwork.oc1.eu-frankfurt-1.xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx is in the VALIDATED state.",
"opc-request-id": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
"status": 409
}