Updated Cloud Deployment Scripts to next version

Author: ChrisJ60

LICENSE.txt

@@ -0,0 +1,25 @@
+You may not use the identified files except in compliance with the Universal Permissive License, Version 1.0 (the License.)
+
+You may obtain a copy of the License at https://oss.oracle.com/licenses/upl. A copy of the license is also reproduced below.
+
+Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+
+See the License for the specific language governing permissions and limitations under the License.
+
+Copyright (c) 2018 Oracle and/or its affiliates.
+
+The Universal Permissive License (UPL), Version 1.0
+
+Subject to the condition set forth below, permission is hereby granted to any person obtaining a copy of this software, associated documentation and/or data (collectively the Software), free of charge and under any and all copyright rights in the Software, and any and all patent rights owned or freely licensable by each licensor hereunder covering either (i) the unmodified Software as contributed to or provided by such licensor, or (ii) the Larger Works (as defined below), to deal in both
+
+(a) the Software, and
+
+(b) any piece of software and/or hardware listed in the lrgrwrks.txt file if one is included with the Software (each a “Larger Work” to which the Software is contributed by such licensors),
+
+without restriction, including without limitation the rights to copy, create derivative works of, display, perform, and distribute the Software and make, use, sell, offer for sale, import, export, have made, and have sold the Software and the Larger Work(s), and to sublicense the foregoing rights on either these or other terms.
+
+This license is subject to the following condition:
+
+The above copyright notice and either this complete permission notice or at a minimum a reference to the UPL must be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED AS IS, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

cloud/ottscaleout/.checksum

@@ -0,0 +1 @@
+31f307103a0b52a4b679aee2ad9e314e

cloud/ottscaleout/LICENSE.md

@@ -0,0 +1,21 @@
+
+
+Terraform and Ansible may be installed from the Development and EPEL repositories available on Oracle's Yum servers.  
+The repositories contain the following note:  
+  Packages For Test and Development  
+  Note: The contents in the following repositories are for development purposes only. Oracle suggests these not be used in production.  
+(http://yum.oracle.com/oracle-linux-7.html)  
+
+Terraform:  
+(https://github.com/hashicorp/terraform/blob/master/LICENSE)  
+
+Ansible:  
+(https://github.com/ansible/ansible/blob/devel/COPYING)  
+
+Terraform Provider OCI:  
+(https://docs.cloud.oracle.com/iaas/Content/API/SDKDocs/terraform.htm?Highlight=license)  
+
+provisionScaloutOCI, underlying Terraform and Ansible scripts for TimesTen Scaleout:  
+Copyright (c) 1999, 2019, Oracle and/or its affiliates. All rights reserved.  
+Licensed under the Universal Permissive License v 1.0 as shown at (http://oss.oracle.com/licenses/upl)  
+

cloud/ottscaleout/README.md

@@ -1,6 +1,6 @@
 ###### Copyright (c) 1999, 2019, Oracle and/or its affiliates. All rights reserved.
 ###### Licensed under the Universal Permissive License v 1.0 as shown at <http://oss.oracle.com/licenses/upl>
-###### Version v3_180412_18.1.2.0.0
+###### Version v4_190716_18.1.0.0.0
     #     ___  ____     _    ____ _     _____
     #    / _ \|  _ \   / \  / ___| |   | ____|
     #   | | | | |_) | / _ \| |   | |   |  _|
@@ -12,16 +12,16 @@ Ansible scripts are provided to rollout a database across the provisioned infras
 This is a bring-your-own-license (BYOL) solution for Oracle TimesTen Scaleout.
 Please see the whitepaper, [Deploying Oracle TimesTen Scaleout Database on OCI](https://www.oracle.com/technetwork/database/database-technologies/timesten/overview/wp-deployingtimestenscaleoutonoci-5069015.pdf "whitepaper")
 
-The example creates a VCN with one or more public subnets running bastion hosts.
-Private subnets are created to run the hosts needed for TimesTen Scaleout.
+The example creates a VCN with one or more regional public subnets running bastion hosts.
+A regional private subnet is created to run the hosts needed for TimesTen Scaleout.
 
 ***
 Deployment diagram.  The dark boxes indicate the default configuration created by the Terraform and Ansible scripts.
-![Deployment diagram](./images/oci-deployment.png "TimesTen Scaleout Deployment Diagram")
+![Deployment diagram](./images/oci-deployment.jpg "TimesTen Scaleout Deployment Diagram")
 
 ***
 
-This distribution provides an utility, _provisionScaleoutOCI_, that automates many provisioning tasks discussed in this README and the associated whitepaper.  If you already have a compute instance provisioned in the OCI cloud, please see the [QUICKSTART.md](./QUICKSTART.md) file.
+This distribution provides an utility, _provisionScaleoutOCI_, that automates many prerequisite tasks discussed in the associated whitepaper.  If you already have an Oracle Linux compute instance provisioned in the OCI cloud, you can use that as a bootstrap instance.  Please see the [QUICKSTART.md](./QUICKSTART.md) file.
 
 ***
 
@@ -39,39 +39,45 @@ This example uses the same set of environment variables as other terraform examp
 To destroy all the resources created
 * terraform destroy -force
 
-The route table used for the private subnets is configured to use a NAT gateway as the default route target.
-Bastion host instances are configured as NAT instances, by enabling forwarding and configuring firewall to do forwarding/masquerading, but routing occurs through the NAT gateway, not through the private IP addresses of the Bastion hosts.
+The route table used for the private subnet is configured to use a NAT gateway as the default route target.
+Bastion host instances are configured as NAT instances, by enabling forwarding and configuring firewall to do forwarding/masquerading.  For high availability purposes routing occurs through the NAT gateway, but can optionally be configured through the private IP addresses of the Bastion hosts.
 See [Using a Private IP as a Route Target](https://docs.us-phoenix-1.oraclecloud.com/Content/Network/Tasks/managingroutetables.htm#privateip) for more details if you wish to use the private IP routing feature.
 
-Once the environment is built, the compute instances on the private network have Internet connectivity.
-A private instance doesn't have a public IP address and it's subnet's route table doesn't contain Internet gateway.  
-You can login into the private instance via ssh from the Bastion host.
-To verify connectivity, you can then run a command like 'ping oracle.com'
+Once the environment is built, the compute instances on the regional private subnets have Internet connectivity.  A private instance doesn't have a public IP address and it's subnet route table doesn't contain Internet gateway.  You can login into the private instance via ssh from the Bastion host.
 
 ### Files in the configuration
 
-#### `env-vars`
+### `provisionScaleoutOCI`
+Utility can be used from an OCI Oracle Linux compute instance to provision TimesTen Scaleout in lieu of manually invoking Terraform and Ansible.  Also aids with meeting prerequisites such as creating ssh credentials and/or using the OCI CLI to create an API Signing key.  See [QUICKSTART.md](./QUICKSTART.md) for more details.  While the utility needs to run on an OCI compute instance, the underlying Terraform and Ansible scripts can be manually invoked from Oracle Linux or macOS systems that can reach the OCI public IP addresses.
+
+### `env-vars`
 Is used to export the environmental variables used in the configuration.
 These variables include references to private key file information for access to the OCI infrastructure.
 
 Before you plan, apply, or destroy the configuration source the file -  
-`$ . env-vars`
+`$ . env-vars`  
+
+### `oci.tf`
+Configures terraform provisioner for oci.  
 
-#### `variables.tf`
+### `variables.tf`
 Defines variables for TimesTen Scaleout configuration.
 Modify this to change the default TimesTen Scaleout 2x2 configuration created.
 
-### `public.tf`
-Creates the VCN, the public subnets, and the compute instances used as bastion hosts.
+### `network.tf`
+Creates the VCN, IG and NAT gateways, route tables, security lists, regional public and private subnets.  
 
-### `private.tf`
-Creates the private subnets, and the compute instances used as zookeeper servers, management instances, or database instances.
+### `compute.tf`
+Creates the compute instances used as bastion hosts, zookeeper servers, management instances, or database instances.  
 
 ### `system-config.tf`
 Contains terraform resources for initial configuration of the compute instances.
-Installs ansible on 1 bastion host.
+Installs ansible on bastion hosts.
 Variables and configuration files for ansible are set up as well.
 
+### `blkvol.tf`
+Optionally creates block volumes.  
+
 ### `service` directory
 * ansible  - Ansible playbooks and task yaml.
 * packages - Required BYOL software: Timesten Scaleout distribution, JDK
@@ -131,8 +137,8 @@ For example, if K==2 and initialAD==2, one data space will be located in AD-2, t
 To use only a single AD for data and management instances
 * Set singleAD == "true" in variables.tf
 
-Bastion hosts are created in AD-1 by default.
-* Set bsInstanceInitialAD=={2|3} to place the bastion host in another AD.  
+Bastion hosts are created starting in AD-1 by default.
+* Set bsInstanceInitialAD=={2|3} to start placing bastion hosts in AD-2 or AD-3.
 This may be useful for trial or pay as you go accounts.  
 For trial or pay-as-you-go accounts set:  
 * initialAD=(bsInitialInstanceAD % 3) + 1
@@ -142,5 +148,37 @@ For trial or pay-as-you-go accounts set:
 To change dbDef (sys.odbc.ini) settings _NOT_ listed in variables.tf, modify the
 service/ansible/roles/mgmtinstances/templates/attributes.dbdef.j2 file.
 
+### Troubleshooting:
 
+Errors encountered during provisioning may be transient ones, such as ssh connectivity or port in use errors.
+Rather than destroying the configuration, Terraform or Ansible can often be rerun to correct such errors.
+Check the output from terraform apply and or ansible for the specific error.
+
+Assuming the (TF_VAR_)service name is 'ttimdb1':  
+
+Setup the environment  
+* `. ./env-vars`  
+If Terraform failed during apply, to rerun Terraform:  
+* `terraform apply --auto-approve`  
+If Terraform succeeds but Ansible failed:  
+Determine the address of the bastion host by running:  
+* `terraform output`  
+Login to the bastion host  
+* `ssh opc@1.2.3.4`  
+Move to ansible directory  
+* `cd service/ansible`  
+Destroy a potentially partially created database, ignoring errors.  
+* `ansible-playbook -i hosts dbdestroy.yaml`  
+Retry Database rollout  
+* `ansible-playbook -i hosts rollout.yaml 2>&1 | tee rollout.out`  
+Check Status  
+* `ansible-playbook -i hosts status.yaml`  
+To view a failed rollout, log into host running mgmt instance.  
+* `ssh -tt [ttimdb1-mg-001|ttimdb1-di-001] sudo su - oracle`  
+* `cat ttimdb1/ttgridrollout.plan`  
+* `cat ttimdb1/ttgridrollout.out`  
+To view database instances
+* `/u10/TimesTen/ttimdb1/iron_mgmt/bin/ttenv ttGridAdmin instancelist`  
+
+Continued failure at this point requires deeper examination of the log files.  
 

cloud/ottscaleout/blkvol.tf

@@ -0,0 +1,96 @@
+# Copyright (c) 1999, 2019, Oracle and/or its affiliates. All rights reserved.
+#
+# Licensed under the Universal Permissive License v 1.0 as shown
+# at http://oss.oracle.com/licenses/upl
+#
+# Block volumes, if requested, are allocated here
+# Block volumes are attached to Standard shapes.
+# A single block volume is allocated per data instance
+# The minimum block volume size is 50GB.
+# Block volumes should minimally be 3X RAM; not checked
+# Commands for {a|de}ttaching volumes via iscsi are generated here
+#
+
+locals {
+
+  # block volumes
+  standard = "${substr(var.diInstanceShape,3,8)}"
+  bvcount1 = "${(local.standard == "Standard") ? 1 : 0}"
+  # validate min/max OCI block volume storage limits or error out
+  bvcount2 = "${(var.diBlockVolumeSizeGB >= 50 && 
+                 var.diBlockVolumeSizeGB <= 32768) ? 
+                local.dicount1 : 0}"
+  # set to indicate error condition if no block volume and Standard shape or size out of range
+  bvcheck1 = "${(local.bvcount2 == 0 && local.bvcount1 == 1) ? 1 : 0}"
+  # mdraid with block volume not supported
+  bvcheck2 = "${(local.bvcount2 == 0 && var.system["storage"] == "MD-RAID-10") ? 1 : 0}"
+
+}
+
+# workaround for error checking
+# prevent use of standard shape without block volume or size out of range
+resource "null_resource" "bv_check_1" {
+  count = "${local.bvcheck1}"
+  provisioner "local-exec" {
+    command = "echo -e '\nERROR: No block volumes or invalid GB specified with Standard shape\nRerun terraform with nonzero value for diBlockVolumeSizeGB\n' && false"
+  }
+}
+
+resource "null_resource" "bv_check_2" {
+  count = "${local.bvcheck2}"
+  provisioner "local-exec" {
+    command = "echo -e '\nERROR: MD-RAID-10 storage not permitted with block volume\nRerun terraform with var.system[\"storage\"] == LVM-READ-0\n' && false"
+  }
+}
+
+# Optional block volume attachments
+# May only used with Standard shape
+
+resource "oci_core_volume" "di_volume" {
+  count = "${local.bvcount2}"
+  #Required
+  availability_domain = "${local.adlist[((count.index % local.numADs) + local.firstAD) % 3]}"
+  compartment_id = "${var.compartment_ocid}"
+
+  #Optional
+  display_name = "${format("%s-bv-%03d", var.service_name, count.index + 1)}"
+  size_in_gbs = "${var.diBlockVolumeSizeGB}"
+  freeform_tags = "${map(var.opc["tagkey"],var.service_name)}"
+}
+
+resource "oci_core_volume_attachment" "di_volume_attachments" {
+  count = "${local.bvcount2 }"
+  #Required
+  instance_id = "${oci_core_instance.di_instance.*.id[count.index]}"
+  attachment_type = "iscsi"
+  volume_id = "${oci_core_volume.di_volume.*.id[count.index]}"
+
+  #Optional
+  display_name = "${format("%s-bvat-%03d", var.service_name, count.index + 1)}"
+
+  provisioner "local-exec" {
+    when = "destroy"
+    command = "rm -rf ${var.opc["scriptdir"]}/iscsi/iscsi-??tach.${self.display_name}"
+  }
+
+  # attach; write iscsi attach commands
+  provisioner "local-exec" {
+    command = "echo iscsiadm -m node -o new -T ${self.iqn} -p ${self.ipv4}:${self.port} >> ${var.opc["scriptdir"]}/iscsi/iscsi-attach.${self.display_name}"
+  }
+  provisioner "local-exec" {
+    command = "echo iscsiadm -m node -o update -n node.startup -v automatic -T ${self.iqn} >> ${var.opc["scriptdir"]}/iscsi/iscsi-attach.${self.display_name}"
+  }
+  provisioner "local-exec" {
+    command = "echo iscsiadm -m node -l -T ${self.iqn} -p ${self.ipv4}:${self.port} >> ${var.opc["scriptdir"]}/iscsi/iscsi-attach.${self.display_name}"
+  }
+
+  # detach; write iscsi detach commands
+  provisioner "local-exec" {
+    command = "echo iscsiadm -m node -u -T ${self.iqn} -p ${self.ipv4}:${self.port} >> ${var.opc["scriptdir"]}/iscsi/iscsi-detach.${self.display_name}"
+  }
+  provisioner "local-exec" {
+    command = "echo iscsiadm -m node -o delete -T ${self.iqn} -p ${self.ipv4}:${self.port} >> ${var.opc["scriptdir"]}/iscsi/iscsi-detach.${self.display_name}"
+  }
+
+}
+