- Configuration Name
- Value
- AZURE_BUCKET_STORAGE_ENABLED
- true
- AZURE_STORAGE_USE_HTTP_PROXY
- false
- AZURE_STORAGE_ACCOUNTNAME
- var.azure_storage_account_name
Specs
- Configuration NameValue
- AZURE_BUCKET_STORAGE_ENABLEDtrue
- AZURE_STORAGE_USE_HTTP_PROXYfalse
- AZURE_STORAGE_ACCOUNTNAMEvar.azure_storage_account_name
- AZURE_STORAGE_BUCKETvar.azure_storage_container_name
- AZURE_STORAGE_ENDPOINT_BASEvar.azure_storage_endpoint_base
- AZURE_STORAGE_OBJECT_KEY_PREFIXvar.name_prefix
- AZURE_STORAGE_REGIONvar.azure_storage_region
- AZURE_STORAGE_ACCOUNTKEYKubernetes Secret: var.k8s_secret_storage_access_key
- AZURE_STORAGE_ENCRYPTION_KEYKubernetes Secret: var.k8s_secret_encryption_key
- KAFKA_COMMON_SECURITY_PROTOCOLSSL
- USING_EPHEMERAL_DISKStrue
- LOCAL_STORAGE_PERCENTAGE80
- LOCAL_STORAGE_MIN_AGE_DAYS1
- KAFKA_BOOTSTRAP_SERVERSvar.kafka_broker_servers
- KAFKA_SERVERSvar.kafka_broker_servers
- PUBLIC_URLhttps://${var.logscale_public_fqdn}
- AUTHENTICATION_METHODstatic
- STATIC_USERSKubernetes Secret: var.k8s_secret_static_user_logins
- KAFKA_COMMON_SSL_TRUSTSTORE_TYPE *PKCS12
- KAFKA_COMMON_SSL_TRUSTSTORE_PASSWORD *Kubernetes Secret: local.kafka_truststore_secret_name
- KAFKA_COMMON_SSL_TRUSTSTORE_LOCATION */tmp/kafka/ca.p12
- NameVersion
- azapi~>1.5
- azurerm~>4.21.0
- helm>=2.13.2,3.4.2
- kubernetes>=2.31.0
- null>=3.2
- random>=3.6.1
- time>=0.9.1
- azure_vnet_address_spaceAddress space to assign to the virtual network that will resources associated to the kubernetes cluster.
- NameDescription
- azure_keyvault_namen/a
- k8s_cluster_idn/a
- k8s_cluster_nameKubernetes information
- k8s_configuration_commandRun this command after building the kubernetes cluster to set your local kube config
- logscale-ingress-fqdnPublic FQDN for access to logscale environment via Azure LB, when public access is enabled
- logscale-ingress-ipPublic IP address for access to logscale environment via Azure LB, when public access is enabled
- resource_group_namen/a
- resource_name_prefixPrefix added to resources for unique identification
README
Contents
LogScale Reference Automations for Azure
This repository contains Terraform configurations to deploy an Azure-based architecture for LogScale. It leverages multiple Azure services including Azure Key Vault, Azure Kubernetes Service, and Azure Storage.
Requirements and Build Information
Prerequisites
Before starting the deployment, ensure you have the following tools and access:
- Terraform 1.10.5: Terraform is the infrastructure as code tool used to manage the deployment. Version 1.10.5 is recommended at this time due to known issues in 1.11.0/1.11.1.
- kubectl 1.232+: kubectl is the command-line tool for interacting with the Kubernetes cluster.
- Azure Command Line 2.68.0+: The Azure Command Line (az cli) allows you to interact with Azure services from the command line.
- Owner Access to Azure Subscription: For full architecture deployment, owner access is expected to the target Azure subscription.
It is additionally recommended but not required to install Helm 3.17.0 or later for troubleshooting helm-based kubernetes deployments.
Azure Access Requirements
The account running this Terraform needs to be assigned the Owner role for the target subscription due to the assignment of roles to the managed identity used by the kubernetes control plane. Role assignment for the Kubernetes cluster is as follows:
- Reader scoped to the Disk Encryption Set created during this process
- Allows identity to read the disk encryption set used for node disk encryption
- Network Contributor scoped to the resource group created by this terraform
- Allows identity to bind a managed load balancer to a public IP created during the Terraform run for environment access
- Key Vault Crypto User scoped to the Azure Key Vault created during this process
- Allows the disk set encryption managed identity the ability to use the key vault for disk encryption
IP-based Access Restrictions
There are four variables that control public access to the environment and set in your TFVAR_FILE configuration.
ip_ranges_allowed_to_kubeapi = ["192.168.3.32/32", "192.168.4.1/32"]
ip_ranges_allowed_https = ["192.168.1.0/24"]
ip_ranges_allowed_to_bastion = ["192.168.3.32/32", "192.168.4.1/32"]
ip_ranges_allowed_kv_access = ["192.168.3.32/32", "192.168.4.1/32"]
- ip_ranges_allowed_to_kubeapi : The Kubernetes API is publicly available by default. This variable limits access to the API and impacts the ability to run Kubernetes API commands.
- ip_ranges_allowed_https : The ingress endpoint for UI Access to Logscale and Ingestion to logscale is publicly available. This limits access.
- ip_ranges_allowed_to_bastion : If you choose to build a bastion host during this process, this limits access to SSH to the host.
- ip_ranges_allowed_kv_access : Access to the Azure Key Vault is limited to ranges defined here.
Note: ip_ranges_allowed_kv_access and ip_ranges_allowed_to_kubeapi must be set correctly for Terraform to operate as expected.
Kubernetes Namespace Separation
Multiple namespaces are created in Kubernetes during the terraform application process in order to promote security and separation of the applications. All namespaces are created using variable var.k8s_namespace_prefix (default: log). Assuming the default value for k8s_namespace_prefix, terraform creates the following namespaces in kubernetes:
- log
- Logscale
- Humio Operator
- Strimzi Kafka Brokers / Controllers (Optional)
- Strimzi Kafka Operator (Optional)
- log-topolvm
- TopoLVM Controller and Nodes
- log-cert
- Cert Manager
- log-ingress
- NGINX ingress controllers
Cluster Size Configuration
The cluster_size.tpl file specifies the available parameters for different sizes of LogScale clusters. This template defines various cluster sizes (e.g., xsmall, small, medium) and their associated configurations, including node counts, instance types, disk sizes, and resource limits. The Terraform configuration uses this template to dynamically configure the LogScale deployment based on the selected cluster size.
-
File:
cluster_size.tpl -
Usage:
The data fromcluster_size.tplis retrieved and rendered by thelocals.tffile. Thelocals.tffile uses thejsondecodefunction to parse the template and select the appropriate cluster size configuration based on thelogscale_cluster_sizevariable. -
Example:
# Local Variables locals { # Render a template of available cluster sizes cluster_size_template = jsondecode(templatefile("${path.module}/cluster_size.tpl", {})) cluster_size_rendered = { for key in keys(local.cluster_size_template) : key => local.cluster_size_template[key] } cluster_size_selected = local.cluster_size_rendered[var.logscale_cluster_size] }
Setting Logscale Configuration Variables
Logscale will be configured with a default set of configuration values that can be overridden or added to by defining var.user_logscale_envars in your TFVAR_FILE. For example, to change default values for LOCAL_STORAGE_MIN_AGE_DAYS and LOCAL_STORAGE_PERCENTAGE, you can set this in your TFVAR_FILE:
user_logscale_envvars = [ { "name" = "LOCAL_STORAGE_MIN_AGE_DAYS", "value" = "7" }, { "name" = "LOCAL_STORAGE_PERCENTAGE", "value" = "85" } ]
This mechanism also supports referencing Kubernetes secrets should you provision them outside this Terraform:
user_logscale_envvars = [
{
"name" = "SECRET_LOGSCALE_CONFIGURATION",
"valueFrom" = {
"secretKeyRef" = {
"key" = "secret_value"
"name" = "kubernetes_secret_name"
}
}
},
{ "name" = "LOCAL_STORAGE_MIN_AGE_DAYS", "value" = "7" },
{ "name" = "LOCAL_STORAGE_PERCENTAGE", "value" = "85" }
]
The default environment values set by this Terraform are as follows:
| Configuration Name | Value |
|---|---|
| AZURE_BUCKET_STORAGE_ENABLED | true |
| AZURE_STORAGE_USE_HTTP_PROXY | false |
| AZURE_STORAGE_ACCOUNTNAME | var.azure_storage_account_name |
| AZURE_STORAGE_BUCKET | var.azure_storage_container_name |
| AZURE_STORAGE_ENDPOINT_BASE | var.azure_storage_endpoint_base |
| AZURE_STORAGE_OBJECT_KEY_PREFIX | var.name_prefix |
| AZURE_STORAGE_REGION | var.azure_storage_region |
| AZURE_STORAGE_ACCOUNTKEY | Kubernetes Secret: var.k8s_secret_storage_access_key |
| AZURE_STORAGE_ENCRYPTION_KEY | Kubernetes Secret: var.k8s_secret_encryption_key |
| KAFKA_COMMON_SECURITY_PROTOCOL | SSL |
| USING_EPHEMERAL_DISKS | true |
| LOCAL_STORAGE_PERCENTAGE | 80 |
| LOCAL_STORAGE_MIN_AGE_DAYS | 1 |
| KAFKA_BOOTSTRAP_SERVERS | var.kafka_broker_servers |
| KAFKA_SERVERS | var.kafka_broker_servers |
| PUBLIC_URL | https://${var.logscale_public_fqdn} |
| AUTHENTICATION_METHOD | static |
| STATIC_USERS | Kubernetes Secret: var.k8s_secret_static_user_logins |
| KAFKA_COMMON_SSL_TRUSTSTORE_TYPE * | PKCS12 |
| KAFKA_COMMON_SSL_TRUSTSTORE_PASSWORD * | Kubernetes Secret: local.kafka_truststore_secret_name |
| KAFKA_COMMON_SSL_TRUSTSTORE_LOCATION * | /tmp/kafka/ca.p12 |
Values marked with * are removed when var.provision_kafka_servers is set to false.
Bring Your Own Kafka
If Kafka already exists and meets the following expectations, it can be used in place of Strimzi created by this Terraform. Expected configuration:
- Client Authentication: None (TBD)
- KRaft Mode: Enabled
- TLS Communications: Enabled
In order to use your own Kafka, make the following modifications to the execution instructions:
- Set terraform variable
provision_kafka_serversto false - Set terraform variable
byo_kafka_connection_stringto your connection string. - Do not execute the build of Strimzi in the following instructions
Bring Your Own Certificate
By default, a Let's Encrypt certificate will be generated and placed on the ingress controller. You can bring your own certificate to the ingress by:
-
Importing or generating a certificate in Azure Keyvault. Reference: Azure Key Vault Certificate Import
-
Passing this information to module logscale in main.tf
module "logscale" { source = "./modules/kubernetes/logscale" use_custom_certificate = true custom_tls_certificate_keyvault_entry = "my-keyvault-item-name" }
A module exists in this Terraform that allows for provisioning a certificate via Azure Key Vault. This is self-signed by default but the module allows for alternative certificate issuers depending on your environment.
module "azure-selfsigned-cert" {
source = "./modules/azure/certificate"
azure_keyvault_id = module.azure-keyvault.keyvault_id
logscale_public_fqdn = "this-is-a-test.local"
name_prefix = local.resource_name_prefix
subject_alternative_names = [module.azure-core.ingress-pub-fqdn, "othername.local"]
cert_issuer = "Self"
}
Targeted Terraform
When leveraging this Terraform repository, you must run terraform using the -target flag to apply specific modules. The latter half of the terraforming process requires access to a kubernetes API to successfully plan and apply changes.
After the environment is fully built, the targeted approach isn't strictly required but remains recommended to ensure proper order of operations.
Terraform Modules
main.tf
This is the core wrapper around all modules provided in this terraform and serves as an example of
how to run this terraform to build your Azure environment.
Requirements
| Name | Version |
|---|---|
| azapi | ~>1.5 |
| azurerm | ~>4.21.0 |
| helm | >=2.13.2,3.4.2 |
| kubernetes | >=2.31.0 |
| null | >=3.2 |
| random | >=3.6.1 |
| time | >=0.9.1 |
Inputs
| Name | Description | Type | Default | Required |
|---|---|---|---|---|
| admin_ssh_pubkey | Your SSH public key for accessing resources via SSH | string |
n/a | yes |
| admin_username | Admin username for ssh access to resources. | string |
"lsroot" |
no |
| aks_azure_policy_enabled | Enable the Azure Policy for AKS add-on allowing for security scanning of kubernetes resources. | bool |
"true" |
no |
| aks_cost_analysis_enabled | Enable cost analysis for this AKS cluster? | bool |
"true" |
no |
| aks_kubernetes_version | Allows specification of the kubernetes version for AKS. Default of 'null' forces use of the latest recommended version at time of provisioning. | string |
null |
no |
| azure_availability_zones | The availability zones to use with your kubernetes cluster. Defaults to null making the cluster regional with no guarantee of HA in the event of zone outage. | list |
null |
no |
| azure_environment | Azure cloud enviroment to use for your resources. Values include: public, usgovernment, german, and china. | string |
"public" |
no |
| azure_keyvault_secret_expiration_days | This ensures that secrets stored in Azure KeyVault expire after X number of days so they are not retained forever. This expiration date will update with every terraform run. | number |
60 |
no |
| azure_resource_group_region | The Azure cloud region for the resource group and associated resources. | string |
n/a | yes |
| azure_subscription_id | Subscription ID where you will build Azure resources. It is expected that you will be Owner of this subscription. | string |
n/a | yes |
| azure_vnet_address_space | Address space to assign to the virtual network that will resources associated to the kubernetes cluster. | list |
[ |
"172.16.0.0/16"
] | no |
| bastion_host_size | Size of virtual machine to launch for bastion host. | string | "Standard_A2_v2" | no |
| byo_kafka_connection_string | Your own kafka environment connection string. | string | "" | no |
| cert_ca_server | Certificate Authority Server. | string | "https://acme-v02.api.letsencrypt.org/directory" | no |
| cert_issuer_email | Certificates issuer email address used with certificates provisioned in the cluster. | string | n/a | yes |
| cert_issuer_kind | Certificates issuer kind for the Logscale cluster. | string | "ClusterIssuer" | no |
| cert_issuer_name | Certificates issuer name for the Logscale Cluster | string | "letsencrypt-cluster-issuer" | no |
| cert_issuer_private_key | This is the kubernetes secret where the private key for the certificate issuer will be stored. | string | "letsencrypt-cluster-issuer-key" | no |
| cm_namespace | Kubernetes namespace used by cert-manager. | string | "cert-manager" | no |
| cm_repo | The cert-manager repository. | string | "https://charts.jetstack.io" | no |
| cm_version | The cert-manager helm chart version | string | n/a | yes |
| diag_logging_eventhub_authorization_rule_id | The rule ID allowing authorization to the eventhub. | string | null | no |
| diag_logging_eventhub_name | The target eventhub name where audit logging will be sent. Use in conjuction with the eventhub_authorization_rule_id | string | null | no |
| diag_logging_loganalytics_id | The ID of the log analytics workspace to send diagnostic logging. | string | null | no |
| diag_logging_storage_account_id | The target storage account id where audit logging will be sent. | string | null | no |
| enable_auditlogging_to_eventhub | Enable audit logging to a target eventhub. | bool | false | no |
| enable_auditlogging_to_loganalytics | Enable audit logging to a target log analytics workspace. | bool | false | no |
| enable_auditlogging_to_storage | Enable audit logging to a target storage account | bool | false | no |
| enable_azure_ddos_protection | Enable DDOS protection for the vnet created by this terraform. Note: DDOS protection will significantly increase the cost of this subscription. | bool | false | no |
| enable_kv_metrics_diag_logging | When sending diagnostic logs for the eventhub resource, we can optionally enable metrics as well. | bool | false | no |
| humio_operator_chart_version | This is the version of the helm chart that installs the humio operator version chosen in variable humio_operator_version. | string | n/a | yes |
| humio_operator_extra_values | Resource Management for logscale pods | map(string) | {
"operator.resources.limits.cpu": "250m",
"operator.resources.limits.memory": "750Mi",
"operator.resources.requests.cpu": "250m",
"operator.resources.requests.memory": "750Mi"
} | no |
| humio_operator_repo | The humio operator repository. | string | "https://humio.github.io/humio-operator" | no |
| humio_operator_version | The humio operator controls provisioning of logscale resources within kubernetes. | string | n/a | yes |
| ip_ranges_allowed_https | List of IP Ranges that can access the ingress frontend for UI and logscale API operations, including ingestion. | list | [] | no |
| ip_ranges_allowed_kv_access | List of IP Ranges that can access the key vault. | list | [] | no |
| ip_ranges_allowed_storage_account_access | IP ranges allowed to access created storage containers | list | [] | no |
| ip_ranges_allowed_to_bastion | (Optional) List of IP addresses or CIDR notated ranges that can access the bastion host. | list(string) | [] | no |
| ip_ranges_allowed_to_kubeapi | IP ranges allowed to access the public kubernetes api | list | [] | no |
| k8s_automatic_upgrade_channel | Upgrade channel for the kubernetes cluster. | string | "patch" | no |
| k8s_config_path | The path that will contain the kubernetes configuration file, typically at ~/.kube/config | string | "~/.kube/config" | no |
| k8s_general_maintenance_windows | This specifies when maintenance operations can be performed on the cluster and will take priority when more specific schedules are not set (i.e. maintenance_window_auto_upgrade, maintenance_window_node_os). | list(object({
day = string
hours = list(number)
})) | [
{
"day": "Sunday",
"hours": [
2,
3,
4
]
}
] | no |
| k8s_maintenance_window_auto_upgrade | Allows for more granular control over AKS auto upgrades | object({
frequency = string # "Weekly", "RelativeMonthly"
interval = number # How often the schedule occurs (e.g., every 1 week/month)
duration = number # Length of maintenance window in hours
day_of_week = string # Required for Weekly frequency
utc_offset = string # e.g., "+00:00", "-07:00"
start_time = string # 24-hour format "HH:mm"
week_index = optional(string) # Required when frequency is RelativeMonthly
}) | null | no |
| k8s_maintenance_window_node_os | Sets a maintenance window for OS upgrades to AKS nodes. | object({
frequency = string # "Weekly", "RelativeMonthly"
interval = number # How often the schedule occurs (e.g., every 1 week/month)
duration = number # Length of maintenance window in hours
day_of_week = string # Required for Weekly / RelativeMonthly frequency
utc_offset = string # e.g., "+00:00", "-07:00"
start_time = string # 24-hour format "HH:mm"
week_index = optional(string) # Required when frequency is RelativeMonthly
}) | null | no |
| k8s_namespace_prefix | Multiple namespaces will be created to contain resources using this prefix. | string | "log" | no |
| k8s_node_os_upgrade_channel | Upgrade channel for the kubernetes nodes. | string | "SecurityPatch" | no |
| kubernetes_private_cluster_enabled | When true, the kubernetes API is only accessible from internal networks (i.e. the bastion host). When false, the API is available to the list of IP ranges provided in variable ip_ranges_allowed_to_kubeapi. | bool | false | no |
| kv_enabled_for_deployment | Allow virtual machines to retrieve certificates stored as secrets in the vault | bool | true | no |
| kv_enabled_for_disk_encryption | Allow azure disk encryption to retrieve and unwrap keys in the vault | bool | true | no |
| kv_purge_protection_enabled | Enable purge protection for KV resources | bool | true | no |
| kv_soft_delete_retention_days | The number of days to retain items once soft-deleted. Values can be 7-90 | number | 7 | no |
| logscale_account_kind | The type of storage account to create. | string | "StorageV2" | no |
| logscale_account_replication | The type of replication to use with the logscale storage account. | string | "LRS" | no |
| logscale_account_tier | Storage account tier. | string | "Standard" | no |
| logscale_cluster_size | Size of the cluster to build in Azure. Reference cluster_size.tpl for definitions. | string | "xsmall" | no |
| logscale_cluster_type | Logscale cluster type | string | n/a | yes |
| logscale_custom_tls_certificate_key_keyvault_name | Name of the TLS certificate key item (PEM format) stored in the Azure Keyvault created by this terraform | string | null | no |
| logscale_custom_tls_certificate_keyvault_name | Name of the TLS certificate item (PEM format) stored in the Azure Keyvault created by this terraform | string | null | no |
| logscale_image | This can be used to specify a full image ref spec. The expectation is that the imagePullSecrets kubernetes secret will exist. | string | null | no |
| logscale_image_version | The version of logscale to install. | string | "" | no |
| logscale_lb_internal_only | The nginx ingress controller to logscale will create a managed azure load balancer with public availability. Setting to true will remove the ability to generate Let's Encrypt certificates in addition to removing public access. | bool | false | no |
| logscale_license | Your logscale license data. | string | n/a | yes |
| logscale_namespace | The kubernetes namespace used by strimzi, logscale, and nginx-ingress. | string | "logging" | no |
| logscale_update_strategy | When describing a HumioCluster resource, you can provide a map value to describe how updates should be applied. Defaults to RollingUpdateBestEffort, 50% maximum unavailable, zone awareness enabled. Reference: https://github.com/humio/humio-operator/blob/master/docs/api.md#humioclusterspecupdatestrategy | map | {
"enableZoneAwareness": true,
"maxUnavailable": "50%",
"minReadySeconds": 120,
"type": "RollingUpdateBestEffort"
} | no |
| network_subnet_aks_ingest_nodes | A list of networks to associate to the ingress node subnet. | list | [
"172.16.5.0/24"
] | no |
| network_subnet_aks_ingress_nodes | A list of networks to associate to the ingress node subnet. | list | [
"172.16.4.0/24"
] | no |
| network_subnet_aks_logscale_digest_nodes | Subnet for the kubernetes node pool hosting logscale digest nodes. | list | [
"172.16.3.0/24"
] | no |
| network_subnet_aks_system_nodes | Subnet for kubernetes system nodes. In the basic architecture, this will also be where nginx ingress nodes are placed. | list | [
"172.16.0.0/24"
] | no |
| network_subnet_aks_ui_nodes | A list of networks to associate to the ingress node subnet. | list | [
"172.16.6.0/24"
] | no |
| network_subnet_bastion_nodes | Subnet for bastion nodes. | list | [
"172.16.1.0/26"
] | no |
| network_subnet_kafka_nodes | Subnet for kubernetes node pool hosting the strimzi kafka nodes | list | [
"172.16.2.0/24"
] | no |
| nginx_ingress_helm_chart_version | The version of nginx-ingress to install in the environment. Reference: github.com/kubernetes/ingress-nginx for helm chart version to nginx version mapping. | string | n/a | yes |
| password_rotation_arbitrary_value | This will not influence the password generated for logscale but, when modified, will cause the password to be regenerated. | string | "defaultstring" | no |
| provision_kafka_servers | Set this to true to provision strimzi kafka within this kubernetes cluster. It should be false if you are bringing your own kafka implementation. | bool | true | no |
| resource_name_prefix | Identifier attached to named resources to help them stand out. Must be 8 or fewer characters which can include lower case, numbers, and hyphens. | string | "log" | no |
| set_kv_expiration_dates | Setting expiration dates on vault secrets will help ensure that secrets are not retained forever but it's not always feasible to have static expiration dates. Set this to false to disable expirations. | bool | true | no |
| strimzi_operator_chart_version | Helm chart version for installing strimzi. | string | n/a | yes |
| strimzi_operator_repo | Strimzi operator repo. | string | "https://strimzi.io/charts/" | no |
| strimzi_operator_version | Strimzi operator version for resource definition installation. | string | n/a | yes |
| tags | A map of tags to apply to all created resources. | map | {} | no |
| topo_lvm_chart_version | Version of topo lvm to install. | string | n/a | yes |
| use_own_certificate_for_ingress | Set to true if you plan to bring your own certificate for logscale ingest/ui access. | bool | false | no |
| user_logscale_envvars | These are environment variables passed into the HumioCluster resource spec definition that will be used for all created logscale instances. Supports string values and kubernetes secret refs. Will override any values defined by default in the configuration. | list(object({
name=string,
value=optional(string)
valueFrom=optional(object({
secretKeyRef = object({
name = string
key = string
})
}))
})) | [] | no |
Outputs
| Name | Description |
|---|---|
| azure_keyvault_name | n/a |
| k8s_cluster_id | n/a |
| k8s_cluster_name | Kubernetes information |
| k8s_configuration_command | Run this command after building the kubernetes cluster to set your local kube config |
| logscale-ingress-fqdn | Public FQDN for access to logscale environment via Azure LB, when public access is enabled |
| logscale-ingress-ip | Public IP address for access to logscale environment via Azure LB, when public access is enabled |
| resource_group_name | n/a |
| resource_name_prefix | Prefix added to resources for unique identification |
Module: azure/core
This module provisions all core requirements for the Azure infrastructure including:
- Azure Virtual Network
- Azure Subnets
- Optional Enablement of DDOS Protection Plan
- NAT Gateway with Public IP
- Public IP and FQDN for Ingress (when public access is enabled)
Inputs
| Name | Description | Type | Default | Required |
|---|---|---|---|---|
| bastion_network_subnet | A list of networks to associate to the bastion host subnet. | list |
n/a | yes |
| enable_azure_ddos_protection | Enable Azure DDOS Protection | bool |
n/a | yes |
| enabled_logscale_digest_service_endpoints | List of service endpoints required for the subnet. Storage is required for vnet-only access. | list |
[ |
"Microsoft.Storage"
] | no |
| environment | Azure cloud enviroment to use for your resources. Values include: public, usgovernment, german, and china. | string | n/a | yes |
| logscale_cluster_type | Type of cluster being built. | string | n/a | yes |
| logscale_lb_internal_only | The nginx ingress controller to logscale will create a managed azure load balancer with public availability. In this core module, this variable determines if the public IP address for this load balancer needs to be created. | bool | false | no |
| name_prefix | Identifier attached to named resources to help them stand out. | string | n/a | yes |
| network_subnet_aks_logscale_digest_nodes | A list of networks to associate to the kubernetes subnet. | list | n/a | yes |
| network_subnet_aks_system_nodes | A list of networks to associate to the kubernetes subnet. | list | n/a | yes |
| network_subnet_ingest_nodes | A list of networks to associate to the ingest node subnet. | list | n/a | yes |
| network_subnet_ingress_nodes | A list of networks to associate to the ingress node subnet. | list | n/a | yes |
| network_subnet_kafka_nodes | A list of networks to associate to the kafka subnet. | list | n/a | yes |
| network_subnet_ui_nodes | A list of networks to associate to the ui node subnet. | list | n/a | yes |
| provision_kafka_servers | Set this to true to provision strimzi kafka within this kubernetes cluster. It should be false if you are bringing your own kafka implementation. | bool | true | no |
| resource_group_region | The Azure cloud region for the resource group and associated resources. | string | n/a | yes |
| subscription_id | Subscription ID for your Azure resources. | string | n/a | yes |
| tags | A map of tags to apply to all created resources. | map | n/a | yes |
| vnet_address_space | Address space to assign to the virtual network that will resources associated to the kubernetes cluster. | list | n/a | yes |
Outputs
| Name | Description |
|---|---|
| bastion_subnet_id | n/a |
| ingress-pub-fqdn | FQDN for logscale ingress when using a public endpoint. |
| ingress-pub-ip | IP Address for logscale ingress when using a public endpoint. |
| ingress-pub-pip-name | n/a |
| ingress-pup-pip-domain-name-label | n/a |
| kafka_nodes_subnet_id | n/a |
| logscale_digest_nodes_subnet_id | n/a |
| logscale_ingest_nodes_subnet_id | n/a |
| logscale_ingress_nodes_subnet_id | n/a |
| logscale_ui_nodes_subnet_id | n/a |
| nat_gw_public_ip | NAT GW IP address for your subnets which can be used to allow access as necessary to other environments. |
| resource_group_id | n/a |
| resource_group_name | Azure Resource Group |
| resource_group_region | n/a |
| system_nodes_subnet_id | n/a |
| vnet_id | n/a |
| vnet_name | n/a |
Module: azure/aks
This module provisions managed Azure Kubernetes within the environment.
Inputs
| Name | Description | Type | Default | Required |
|---|---|---|---|---|
| admin_ssh_pubkey | Public key for SSH access to the bastion host. | string |
n/a | yes |
| admin_username | Admin username for ssh access to k8s nodes. | string |
n/a | yes |
| aks_system_nodes_subnet_id | Subnet ID for AKS system nodes to live in. | any |
n/a | yes |
| authorized_ip_ranges | IP Ranges allowed to access the public kubernetes API | list |
[] |
no |
| azure_availability_zones | The availability zones to use with your kubernetes cluster. Defaults to null making the cluster regional with no guarantee of HA in the event of zone outage. | list |
null |
no |
| azure_keyvault_id | Azure KeyVault id used for storing secrets related to this infrastructure | string |
n/a | yes |
| azure_keyvault_secret_expiration_date | When secrets should expire. | string |
n/a | yes |
| azure_policy_enabled | Enable the Azure Policy for AKS add-on? | bool |
n/a | yes |
| cost_analysis_enabled | Enable cost analysis for this AKS cluster? | bool |
n/a | yes |
| diag_logging_eventhub_authorization_rule_id | The rule ID allowing authorization to the eventhub. | string |
null |
no |
| diag_logging_eventhub_name | The target eventhub name where audit logging will be sent. Use in conjuction with the eventhub_authorization_rule_id | string |
null |
no |
| diag_logging_loganalytics_id | The ID of the log analytics workspace to send diagnostic logging. | string |
null |
no |
| diag_logging_storage_account_id | The target storage account id where audit logging will be sent. | string |
null |
no |
| disk_encryption_key_expiration_date | Optionally set when the disk encryption key used for AKS nodes should expire. Defaults to null on the assumption that this AKS cluster might be long-lived. | string |
null |
no |
| enable_auditlogging_to_eventhub | Enable audit logging to a target eventhub. | bool |
false |
no |
| enable_auditlogging_to_loganalytics | Enable audit logging to a target log analytics workspace. | bool |
false |
no |
| enable_auditlogging_to_storage | Enable audit logging to a target storage account | bool |
false |
no |
| enable_kv_metrics_diag_logging | When sending diagnostic logs for the eventhub resource, we can optionally enable metrics as well. | bool |
false |
no |
| environment | Azure cloud enviroment to use for your resources. | string |
n/a | yes |
| ip_ranges_allowed_https | IP Ranges allowed to access the nginx-ingress loadbalancer pods | list |
[] |
no |
| k8s_automatic_upgrade_channel | Upgrade channel for the kubernetes cluster. | string |
"patch" |
no |
| k8s_general_maintenance_windows | This specifies when maintenance operations can be performed on the cluster and will take priority when more specific schedules are not set (i.e. maintenance_window_auto_upgrade, maintenance_window_node_os). | list(object({ |
day = string
hours = list(number)
})) | [
{
"day": "Sunday",
"hours": [
2,
3,
4
]
}
] | no |
| k8s_maintenance_window_auto_upgrade | Allows for more granular control over AKS auto upgrades | object({
frequency = string # "Weekly", "RelativeMonthly"
interval = number # How often the schedule occurs (e.g., every 1 week/month)
duration = number # Length of maintenance window in hours
day_of_week = string # Required for Weekly frequency
utc_offset = string # e.g., "+00:00", "-07:00"
start_time = string # 24-hour format "HH:mm"
week_index = optional(string) # Required when frequency is RelativeMonthly
}) | null | no |
| k8s_maintenance_window_node_os | Sets a maintenance window for OS upgrades to AKS nodes. | object({
frequency = string # "Weekly", "RelativeMonthly"
interval = number # How often the schedule occurs (e.g., every 1 week/month)
duration = number # Length of maintenance window in hours
day_of_week = string # Required for Weekly / RelativeMonthly frequency
utc_offset = string # e.g., "+00:00", "-07:00"
start_time = string # 24-hour format "HH:mm"
week_index = optional(string) # Required when frequency is RelativeMonthly
}) | null | no |
| k8s_node_os_upgrade_channel | Upgrade channel for the kubernetes nodes. | string | "SecurityPatch" | no |
| kafka_nodes_subnet_id | Subnet ID where kafka nodes will live. | string | n/a | yes |
| kubernetes_diagnostic_log_categories | List of enabled diagnostic log categories for the kubernetes cluster. | list | [
"kube-apiserver",
"kube-controller-manager",
"kube-scheduler",
"kube-audit",
"kube-audit-admin"
] | no |
| kubernetes_version | Allows specification of the kubernetes version for AKS. Default of 'null' forces use of the latest recommended version at time of provisioning. | string | null | no |
| logscale_cluster_type | Logscale cluster type | string | n/a | yes |
| logscale_digest_nodes_subnet_id | Subnet ID for logscale digest nodes. | any | n/a | yes |
| logscale_ingest_node_desired_count | n/a | number | n/a | yes |
| logscale_ingest_node_max_count | n/a | number | n/a | yes |
| logscale_ingest_node_min_count | n/a | number | n/a | yes |
| logscale_ingest_nodes_subnet_id | Subnet ID for ingest nodes. | any | n/a | yes |
| logscale_ingest_os_disk_size | n/a | number | n/a | yes |
| logscale_ingest_vmsize | n/a | string | n/a | yes |
| logscale_ingress_node_desired_count | n/a | number | n/a | yes |
| logscale_ingress_node_max_count | n/a | number | n/a | yes |
| logscale_ingress_node_min_count | n/a | number | n/a | yes |
| logscale_ingress_nodes_subnet_id | Subnet ID for ingest nodes. | any | n/a | yes |
| logscale_ingress_os_disk_size | n/a | number | n/a | yes |
| logscale_ingress_vmsize | n/a | string | n/a | yes |
| logscale_node_desired_count | n/a | number | n/a | yes |
| logscale_node_max_count | n/a | number | n/a | yes |
| logscale_node_min_count | n/a | number | n/a | yes |
| logscale_node_os_disk_size_gb | n/a | number | n/a | yes |
| logscale_node_vmsize | n/a | string | n/a | yes |
| logscale_ui_node_desired_count | n/a | number | n/a | yes |
| logscale_ui_node_max_count | n/a | number | n/a | yes |
| logscale_ui_node_min_count | n/a | number | n/a | yes |
| logscale_ui_nodes_subnet_id | Subnet ID for ingest nodes. | any | n/a | yes |
| logscale_ui_os_disk_size | n/a | number | n/a | yes |
| logscale_ui_vmsize | n/a | string | n/a | yes |
| name_prefix | Identifier attached to named resources to help them stand out. | string | n/a | yes |
| private_cluster_enabled | Should the kubernetes API be private only? Setting to private has implications to how to run this IaaC. Refer to documentation for more detail. | bool | n/a | yes |
| provision_kafka_servers | Set this to true to provision strimzi kafka within this kubernetes cluster. It should be false if you are bringing your own kafka implementation. | bool | true | no |
| resource_group_id | The ID of the resource group where the kubernetes managed identity will be granted network contributor access. | string | n/a | yes |
| resource_group_name | The Azure cloud region for the resource group and associated resources. | string | n/a | yes |
| resource_group_region | The Azure cloud region for the resource group and associated resources. | string | n/a | yes |
| sku_tier | Tier for the AKS cluster, Standard or Premium | string | "Standard" | no |
| strimzi_node_desired_count | n/a | number | n/a | yes |
| strimzi_node_instance_type | n/a | string | n/a | yes |
| strimzi_node_max_count | n/a | number | n/a | yes |
| strimzi_node_min_count | n/a | number | n/a | yes |
| strimzi_node_os_disk_size_gb | n/a | number | n/a | yes |
| subscription_id | Subscription ID for your Azure resources. | string | n/a | yes |
| system_node_desired_count | n/a | number | n/a | yes |
| system_node_max_count | n/a | number | n/a | yes |
| system_node_min_count | n/a | number | n/a | yes |
| system_node_os_disk_size_gb | n/a | number | n/a | yes |
| system_node_vmsize | n/a | string | n/a | yes |
| tags | A map of tags to apply to all created resources. | map | n/a | yes |
| use_custom_certificate | Use a custom provided certificate for ingress. In this module, this setting controls creation of a NSG rule that allows for Let's Encrypt ACME challenges. | bool | false | no |
Outputs
| Name | Description |
|---|---|
| k8s_client_certificate_kv_name | n/a |
| k8s_client_key_kv_name | n/a |
| k8s_cluster_ca_certificate_kv_name | n/a |
| k8s_cluster_endpoint | n/a |
| k8s_cluster_id | n/a |
| k8s_cluster_name | n/a |
| k8s_cluster_principal_id | Kubernetes information |
| k8s_kube_config_kv_name | n/a |
Module: azure/bastion
An optional module that can be used to provision a bastion host. This is particularly useful when provisioning a brand new
environment and setting the kubernetes API to private access only.
Inputs
| Name | Description | Type | Default | Required |
|---|---|---|---|---|
| admin_ssh_pubkey | Public key for SSH access to the bastion host. | string |
n/a | yes |
| admin_username | Admin username for ssh access to k8s nodes. | string |
n/a | yes |
| bastion_host_size | Sizing for the bastion host. | string |
n/a | yes |
| bastion_subnet_id | Subnet ID to attach the bastion host NIC. | string |
n/a | yes |
| environment | Azure cloud enviroment to use for your resources. | string |
n/a | yes |
| ip_ranges_allowed | List of IP addresses or CIDR notated ranges that can access the bastion host. | list |
n/a | yes |
| name_prefix | Identifier attached to named resources to help them stand out. | string |
n/a | yes |
| resource_group_name | The Azure cloud region for the resource group and associated resources. | string |
n/a | yes |
| resource_group_region | The Azure cloud region for the resource group and associated resources. | string |
n/a | yes |
| subscription_id | Subscription ID for your Azure resources. | string |
n/a | yes |
| tags | A map of tags to apply to all created resources. | map |
n/a | yes |
| vnet_name | Name of the virtual network where this resource will live | string |
n/a | yes |
Outputs
| Name | Description |
|---|---|
| bastion_host_private_ip | n/a |
| bastion_nsg_name | n/a |
| bastion_public_dns_name | n/a |
| bastion_public_ip_address | Bastion Host Connection Information |
Module: azure/certificate
An optional module that can be used to provision a certificate within Azure KeyVault. This is expected to be used for self-signed
test certificates but depending on the configuration of your KeyVault, it can be leveraged to provsion valid certs.
Inputs
| Name | Description | Type | Default | Required |
|---|---|---|---|---|
| azure_keyvault_id | The Azure Keyvault ID storing all the secrets above. | string |
n/a | yes |
| cert_issuer | The issuer to use for certificate generation. Defaults to Self but can match any issuer registered in your environment. | string |
"Self" |
no |
| logscale_public_fqdn | The FQDN tied to the public IP address for logscale ingress. This is the resource that will have a certificate provisioned from let's encrypt. | string |
n/a | yes |
| name_prefix | Identifier attached to named resources to help them stand out. | string |
n/a | yes |
| subject_alternative_names | List of alternative names for the certificate. | list |
[] |
no |
Outputs
| Name | Description |
|---|---|
| certificate_keyvault_name | n/a |
Module: azure/identity
This optional module can be used to provision a managed identity in Azure and assign a role to the identity.
Inputs
| Name | Description | Type | Default | Required |
|---|---|---|---|---|
| name_prefix | Identifier attached to named resources to help them stand out. | string |
n/a | yes |
| resource_group_name | The Azure cloud region for the resource group and associated resources. | string |
n/a | yes |
| resource_group_region | The Azure cloud region for the resource group and associated resources. | string |
n/a | yes |
| role_definition_name | Built-in role definition to assign to the created identity | string |
"Storage Blob Data Owner" |
no |
| tags | A map of tags to apply to all created resources. | map |
n/a | yes |
Outputs
| Name | Description |
|---|---|
| managed_identity_resource_id | n/a |
| managed_identity_resource_principal_id | n/a |
Module: azure/keyvault
This module provisions an Azure Keyvault for storing sensititve environment secrets for use across terraform modules.
Inputs
| Name | Description | Type | Default | Required |
|---|---|---|---|---|
| certificate_permissions | The keyvault will be created with an access policy that grants permission to the calling user to do most things for the purposes of this terraform run. These permissions can be adjusted as required. | list |
[ |
"Create",
"Delete",
"Get",
"GetIssuers",
"Import",
"List",
"ListIssuers",
"Update"
] | no |
| diag_logging_eventhub_authorization_rule_id | The rule ID allowing authorization to the eventhub. | string | null | no |
| diag_logging_eventhub_name | The target eventhub name where audit logging for the keyvault will be sent. Use in conjuction with the eventhub_authorization_rule_id | string | null | no |
| diag_logging_loganalytics_id | The ID of the log analytics workspace to send diagnostic logging. | string | null | no |
| diag_logging_storage_account_id | The target storage account id where audit logging for the keyvault will be sent. | string | null | no |
| enable_auditlogging_to_eventhub | Enable audit logging to a target eventhub. | bool | false | no |
| enable_auditlogging_to_loganalytics | Enable audit logging to a target log analytics workspace. | bool | false | no |
| enable_auditlogging_to_storage | Enable audit logging to a target storage account | bool | false | no |
| enable_kv_metrics_diag_logging | When sending diagnostic logs for the eventhub resource, we can optionally enable metrics as well. | bool | false | no |
| enabled_for_deployment | Allow virtual machines to retrieve certificates stored as secrets in the vault | bool | true | no |
| enabled_for_disk_encryption | Allow azure disk encryption to retrieve and unwrap keys in the vault | bool | true | no |
| ip_ranges_allowed_kv_access | IP Ranges allowed access to keyvault outside of trusted AzureServices. | list | [] | no |
| key_permissions | The keyvault will be created with an access policy that grants permission to the calling user to do most things for the purposes of this terraform run. These permissions can be adjusted as required. | list | [
"Get",
"List",
"Update",
"Delete",
"Encrypt",
"Decrypt",
"WrapKey",
"UnwrapKey",
"Create",
"GetRotationPolicy",
"SetRotationPolicy",
"Recover",
"Purge",
"Backup"
] | no |
| name_prefix | Identifier attached to named resources to help them stand out. | string | n/a | yes |
| purge_protection_enabled | Enable purge protection for KV resources | bool | true | no |
| resource_group_name | The Azure cloud region for the resource group and associated resources. | string | n/a | yes |
| resource_group_region | The Azure cloud region for the resource group and associated resources. | string | n/a | yes |
| secret_permissions | The keyvault will be created with an access policy that grants permission to the calling user to do most things for the purposes of this terraform run. These permissions can be adjusted as required. | list | [
"Get",
"List",
"Set",
"Delete",
"Recover"
] | no |
| sku_name | Standard or Premium SKU for the key vault. | string | "standard" | no |
| soft_delete_retention_days | The number of days to retain items once soft-deleted. Values can be 7-90 | number | 7 | no |
| tags | A map of tags to apply to all created resources. | map | n/a | yes |
Outputs
| Name | Description |
|---|---|
| keyvault_id | n/a |
| keyvault_name | n/a |
Module: azure/storage
This module provisions a storage account and optional container designed for provisioning storage for Logscale Object storage.
Inputs
| Name | Description | Type | Default | Required |
|---|---|---|---|---|
| azure_keyvault_id | Azure KeyVault id used for storing secrets related to this infrastructure | string |
n/a | yes |
| azure_keyvault_secret_expiration_date | When secrets should expire. | string |
n/a | yes |
| create_container | n/a | bool |
false |
no |
| diag_logging_eventhub_authorization_rule_id | The rule ID allowing authorization to the eventhub. | string |
null |
no |
| diag_logging_eventhub_name | The target eventhub name where audit logging will be sent. Use in conjuction with the eventhub_authorization_rule_id | string |
null |
no |
| diag_logging_loganalytics_id | The ID of the log analytics workspace to send diagnostic logging. | string |
null |
no |
| diag_logging_storage_account_id | The target storage account id where audit logging will be sent. | string |
null |
no |
| enable_auditlogging_to_eventhub | Enable audit logging to a target eventhub. | bool |
false |
no |
| enable_auditlogging_to_loganalytics | Enable audit logging to a target log analytics workspace. | bool |
false |
no |
| enable_auditlogging_to_storage | Enable audit logging to a target storage account | bool |
false |
no |
| enable_hns | n/a | bool |
true |
no |
| ip_ranges_allowed_storage_account_access | IP Ranges allowed access to the storage account | list |
[] |
no |
| min_tls_version | Minimum TLS version accepted by the storage container. | string |
"TLS1_2" |
no |
| name_prefix | Identifier attached to named resources to help them stand out. | string |
n/a | yes |
| resource_group_name | The Azure cloud region for the resource group and |
