Apache CloudStack Cloud Usage
This document presents the concept of cloud computing environments, under
the context of private cloud, and a introduction to the Apache CloudStack cloud
orchestration platform, focusing on its usage process.
Last update: February 24, 2025
Revision:
Contents
1 Introduction 6
1.1 Private cloud platforms . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.1.1 Basic functionalities . . . . . . . . . . . . . . . . . . . . . . . 7
1.2 Apache CloudStack history . . . . . . . . . . . . . . . . . . . . . . . 9
2 Apache CloudStack functionalities 10
2.1 Home dashboard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.2 Managing access to the cloud . . . . . . . . . . . . . . . . . . . . . . 10
2.2.1 Account . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.2.1.1 Adding an account . . . . . . . . . . . . . . . . . . . 13
2.2.1.2 Visualizing account details . . . . . . . . . . . . . . 14
2.2.1.3 Removing an account . . . . . . . . . . . . . . . . . 15
2.2.1.4 Configuring account accesses . . . . . . . . . . . . 15
2.2.1.5 Disabling an account . . . . . . . . . . . . . . . . . 17
2.2.1.6 Blocking an account . . . . . . . . . . . . . . . . . . 18
2.2.1.7 API access restriction by IP . . . . . . . . . . . . . . 19
2.2.2 Roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.2.2.1 Adding a new role . . . . . . . . . . . . . . . . . . . 20
2.2.2.2 Visualizing role details . . . . . . . . . . . . . . . . . 21
2.2.2.3 Customizing a role . . . . . . . . . . . . . . . . . . . 22
2.2.2.4 Removing a role . . . . . . . . . . . . . . . . . . . . 22
2.2.3 Domains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.2.3.1 Adding a domain . . . . . . . . . . . . . . . . . . . . 25
2.2.3.2 Configuring the domain . . . . . . . . . . . . . . . . 26
2.2.3.3 Defining the domain accesses . . . . . . . . . . . . 27
2.2.3.4 Restructuring the domain tree . . . . . . . . . . . . 28
2.2.4 Users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.2.4.1 Adding an user . . . . . . . . . . . . . . . . . . . . . 29
2
2.2.4.2 Editing a user . . . . . . . . . . . . . . . . . . . . . . 31
2.2.4.3 Deleting an user . . . . . . . . . . . . . . . . . . . . 32
2.2.4.4 Disabling/Enabling an user . . . . . . . . . . . . . . 33
2.2.4.5 Paswword policies . . . . . . . . . . . . . . . . . . . 34
2.2.4.6 Disabling the user for wrong password . . . . . . . 36
2.2.4.7 Managing user API keys . . . . . . . . . . . . . . . . 36
2.2.4.8 API keys usage in CloudMonkey . . . . . . . . . . . 38
2.2.4.9 Two factors authentication . . . . . . . . . . . . . . 38
2.2.4.10 Timezone . . . . . . . . . . . . . . . . . . . . . . . . 43
2.2.5 Projects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
2.2.5.1 adding a project . . . . . . . . . . . . . . . . . . . . 44
2.2.5.2 Adding accounts and users to the project . . . . . 48
2.2.5.3 Editing a project . . . . . . . . . . . . . . . . . . . . 52
2.2.5.4 Suspending a project . . . . . . . . . . . . . . . . . 52
2.2.5.5 Deleting a project . . . . . . . . . . . . . . . . . . . 53
2.2.6 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
2.3 VMs management . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
2.3.1 Adding a VM . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
2.3.1.1 Adding a VM as root admin . . . . . . . . . . . . . . 56
2.3.1.2 Adding a VM as an admin . . . . . . . . . . . . . . . 58
2.3.1.3 Adding a VM as an user . . . . . . . . . . . . . . . . 60
2.3.1.4 Creating a VM with UEFI initialization . . . . . . . . 61
2.3.2 Removing a VM . . . . . . . . . . . . . . . . . . . . . . . . . . 65
2.3.3 Starting and stopping VMs . . . . . . . . . . . . . . . . . . . 67
2.3.4 Restarting a VM . . . . . . . . . . . . . . . . . . . . . . . . . . 67
2.3.5 Accessing Vms via web interface . . . . . . . . . . . . . . . . 67
2.3.6 Increasing computational resources from a VM . . . . . . . 68
2.3.7 Assigning a VM to another account . . . . . . . . . . . . . . 71
2.3.8 Adding user-data in a VM . . . . . . . . . . . . . . . . . . . . 73
3
2.3.9 VM settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
2.3.9.1 General settings . . . . . . . . . . . . . . . . . . . . 76
2.3.9.2 Settings for VMs with custom compute offering . 76
2.3.9.3 Miscellaneous settings for internal usage . . . . . 77
2.3.9.4 Settings for importing VMs with NIC, disk and cus-
tom parameters for custom compute offerings . . 77
2.3.9.5 Adding VM settings via API . . . . . . . . . . . . . . 77
2.4 Managing volumes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
2.4.1 Operations in data disks and root disks . . . . . . . . . . . . 79
2.4.1.1 Viewing disks and volumes . . . . . . . . . . . . . . 79
2.4.1.2 Creating a volume . . . . . . . . . . . . . . . . . . . 80
2.4.1.3 Uploading a local volume . . . . . . . . . . . . . . . 81
2.4.1.4 Uploading a local volume using cURL . . . . . . . . 82
2.4.1.5 Online volume upload via URL . . . . . . . . . . . . 83
2.4.1.6 Adding a volume to a VM . . . . . . . . . . . . . . . 84
2.4.1.7 Removing a volume from a VM . . . . . . . . . . . 85
2.4.1.8 Removing and adding a ROOT volume . . . . . . . 85
2.4.1.9 Resizing a data disk volume . . . . . . . . . . . . . 86
2.4.1.10 Volume download . . . . . . . . . . . . . . . . . . . 87
2.4.1.11 Recurring snapshots . . . . . . . . . . . . . . . . . . 88
2.4.1.12 Removing a volume . . . . . . . . . . . . . . . . . . 89
2.5 Guest networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
2.5.1 Operations with guest networks . . . . . . . . . . . . . . . . 90
2.5.1.1 Add a guest network . . . . . . . . . . . . . . . . . 90
2.5.1.2 Network types . . . . . . . . . . . . . . . . . . . . . 90
2.5.1.3 Acesso aos detalhes de uma rede guest . . . . . . 96
2.5.1.4 Removing networks . . . . . . . . . . . . . . . . . . 96
2.5.1.5 Editing a network . . . . . . . . . . . . . . . . . . . 97
2.5.1.6 Restarting a network . . . . . . . . . . . . . . . . . 97
4
2.5.1.7 Egress rules . . . . . . . . . . . . . . . . . . . . . . . 97
2.5.2 Network permissions . . . . . . . . . . . . . . . . . . . . . . . 98
2.6 Public IPs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
2.6.1 Operations with public IPs . . . . . . . . . . . . . . . . . . . 102
2.6.1.1 Firewall setup . . . . . . . . . . . . . . . . . . . . . . 102
2.6.1.2 Enabling static NAT . . . . . . . . . . . . . . . . . . 103
2.6.1.3 Enabling port forwarding . . . . . . . . . . . . . . . 105
2.6.1.4 Enabling load balancing . . . . . . . . . . . . . . . . 106
2.6.2 Public IP acquisition . . . . . . . . . . . . . . . . . . . . . . . 108
2.7 Virtual Private Cloud (VPC) . . . . . . . . . . . . . . . . . . . . . . . . 110
2.7.1 VPC operations . . . . . . . . . . . . . . . . . . . . . . . . . . 111
2.7.1.1 Adding a VPC . . . . . . . . . . . . . . . . . . . . . . 111
2.7.1.2 Restarting a VPC . . . . . . . . . . . . . . . . . . . . 113
2.7.1.3 Removing a VPC . . . . . . . . . . . . . . . . . . . . 114
2.7.1.4 Public IP acquisition for a VPC . . . . . . . . . . . . 114
2.7.2 Tiers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
2.7.2.1 Adding tiers to a VPC . . . . . . . . . . . . . . . . . 116
2.7.2.2 Removing tiers from a VPC: . . . . . . . . . . . . . 118
2.7.3 Network ACL . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
2.7.4 Domain VPC . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
2.7.4.1 Allocating public IPs in domain VPCs . . . . . . . . 123
2.8 Virtual Private Network (VPN) . . . . . . . . . . . . . . . . . . . . . . 124
2.8.1 VPN types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
2.8.2 Operations with VPNs . . . . . . . . . . . . . . . . . . . . . . 125
2.8.2.1 Creating a VPN . . . . . . . . . . . . . . . . . . . . . 125
2.8.2.2 Disabling a VPN for a VPC . . . . . . . . . . . . . . . 127
2.8.2.3 Managing users . . . . . . . . . . . . . . . . . . . . 128
2.8.2.4 Adding a user . . . . . . . . . . . . . . . . . . . . . . 128
2.8.2.5 Defining maximum VPN users per account . . . . 129
5
2.8.2.6 Removing an user . . . . . . . . . . . . . . . . . . . 129
2.8.2.7 Connecting to the VPN via Linux . . . . . . . . . . . 130
2.8.2.8 Connecting to the VPN via Windows . . . . . . . . 132
2.8.3 S2S (Site-to-Site) VPNs . . . . . . . . . . . . . . . . . . . . . . 136
2.9 Signing HTTP requests . . . . . . . . . . . . . . . . . . . . . . . . . . 141
2.10 Templates and ISOs . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
2.10.1 Operations with templates/ISOs . . . . . . . . . . . . . . . . 147
2.10.1.1 Viewing . . . . . . . . . . . . . . . . . . . . . . . . . 148
2.10.1.2 Listing . . . . . . . . . . . . . . . . . . . . . . . . . . 150
2.10.1.3 Registry . . . . . . . . . . . . . . . . . . . . . . . . . 152
2.10.1.4 Upload . . . . . . . . . . . . . . . . . . . . . . . . . . 155
2.10.1.5 Editing . . . . . . . . . . . . . . . . . . . . . . . . . . 156
2.10.1.6 Editing template/ISO permissions . . . . . . . . . . 158
2.10.1.7 Editing the template sharing . . . . . . . . . . . . . 160
2.10.1.8 Removing a template/ISO . . . . . . . . . . . . . . . 161
2.10.1.9 Creating a template based in a existing VM . . . . 162
2.11 Resource Tags . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
2.12 Affinity Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
2.12.1 Affinity groups types . . . . . . . . . . . . . . . . . . . . . . . 165
2.12.2 Affinity groups operations . . . . . . . . . . . . . . . . . . . . 166
2.12.2.1 Adding an affinity group . . . . . . . . . . . . . . . 166
2.12.2.2 Adding a VM to an affinity group . . . . . . . . . . 167
2.12.2.3 Viewing VMs in the affinity group . . . . . . . . . . 168
2.12.2.4 Removing an affinity groups . . . . . . . . . . . . . 170
2.13 Autoscale VM groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
2.13.1 Creating am autoscale VM group . . . . . . . . . . . . . . . . 171
2.13.1.1 Setting up a scale-down policy . . . . . . . . . . . . 175
2.13.1.2 Configuring autoscale VM group details . . . . . . 176
2.13.1.3 Group creation . . . . . . . . . . . . . . . . . . . . . 177
6
2.13.2 Testing the autoscaling . . . . . . . . . . . . . . . . . . . . . 178
2.13.3 Editing an autoscale VM group . . . . . . . . . . . . . . . . . 179
2.13.4 Removing an autoscale VM group . . . . . . . . . . . . . . . 182
2.14 Kubernetes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
2.14.1 Kubernetes versions supported . . . . . . . . . . . . . . . . 184
2.14.2 Adding a Kubernetes ISO to the Apache CloudStack . . . . 184
2.14.3 Removing a Kubernetes ISO . . . . . . . . . . . . . . . . . . 186
2.14.4 Kubernetes clusters . . . . . . . . . . . . . . . . . . . . . . . 186
2.14.4.1 Creating Kubernetes clusters . . . . . . . . . . . . 187
2.14.4.2 Scaling of Kubernetes cluster . . . . . . . . . . . . 189
2.14.4.3 Upgrading a Kubernetes cluster . . . . . . . . . . . 190
2.14.5 Kubernetes access . . . . . . . . . . . . . . . . . . . . . . . . 190
2.14.6 Generating a token for the dashboard . . . . . . . . . . . . 192
2.15 Snapshots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
2.15.1 Snapshot types . . . . . . . . . . . . . . . . . . . . . . . . . . 192
2.15.1.1 Volume/Disk snapshot . . . . . . . . . . . . . . . . 192
2.15.1.2 VM snapshot . . . . . . . . . . . . . . . . . . . . . . 193
2.15.2 Snapshot operations . . . . . . . . . . . . . . . . . . . . . . . 193
2.15.2.1 Volume snapshot . . . . . . . . . . . . . . . . . . . 194
2.15.2.2 Vm snapshot . . . . . . . . . . . . . . . . . . . . . . 195
2.15.2.3 Viewing created snapshots . . . . . . . . . . . . . . 196
2.15.2.4 Creating a template from a volume snapshot . . . 197
2.15.2.5 Creating a volume from a volume snapshot . . . . 198
2.15.2.6 Reverting a volume snapshot . . . . . . . . . . . . 199
2.15.2.7 Removing a volume snapshot . . . . . . . . . . . . 200
2.15.2.8 Creating a volume snapshot from a VM snapshot 200
2.15.2.9 Reverting a VM snapshot . . . . . . . . . . . . . . . 202
2.15.2.10Removing a VM snapshot . . . . . . . . . . . . . . . 202
2.15.2.11Configuring recurrent snapshots . . . . . . . . . . 203
7
2.16 Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
2.16.1 Event types . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
2.16.2 Searching and filtering events . . . . . . . . . . . . . . . . . 203
2.16.3 Removing or archiving events . . . . . . . . . . . . . . . . . 204
2.17 User interface (UI) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
2.17.1 Using the browser’s timezone . . . . . . . . . . . . . . . . . 205
2.17.2 UI documentation . . . . . . . . . . . . . . . . . . . . . . . . 206
2.18 Service offerings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
2.18.1 Disk offerings . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
2.18.1.1 Creating a disk offering . . . . . . . . . . . . . . . . 207
2.18.1.2 Editing a disk offering . . . . . . . . . . . . . . . . . 210
2.18.1.3 Removing a disk offering . . . . . . . . . . . . . . . 212
2.18.2 Compute offerings . . . . . . . . . . . . . . . . . . . . . . . . 213
2.18.2.1 Creating a compute offering . . . . . . . . . . . . . 213
2.18.2.2 Editing a compute offering . . . . . . . . . . . . . . 218
2.18.2.3 Removing a compute offering . . . . . . . . . . . . 219
2.18.2.4 Changing the compute offering of a VM . . . . . . 220
2.18.3 Network offerings . . . . . . . . . . . . . . . . . . . . . . . . 222
2.18.3.1 Creating a network offering . . . . . . . . . . . . . 222
2.18.3.2 Editing a network offering . . . . . . . . . . . . . . 226
2.18.3.3 Removing a network offering . . . . . . . . . . . . 228
2.18.4 VPC offering . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
2.18.4.1 Creating a VPC offering . . . . . . . . . . . . . . . . 229
2.18.4.2 Editing a VPC offering . . . . . . . . . . . . . . . . . 230
2.18.4.3 Removing a VPC offering . . . . . . . . . . . . . . . 232
3 CloudMonkey, API and others 234
3.1 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234
3.2 Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234
3.3 Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
8
4 Resources consumption accounting 240
4.1 Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240
4.1.1 Resource usage report . . . . . . . . . . . . . . . . . . . . . . 240
4.2 Quota . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241
4.2.1 Tariffs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241
4.2.2 Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241
4.2.2.1 Credits . . . . . . . . . . . . . . . . . . . . . . . . . . 242
4.2.2.2 Resource usage . . . . . . . . . . . . . . . . . . . . 243
4.2.2.3 Balance . . . . . . . . . . . . . . . . . . . . . . . . . 246
5 Cloud-init 249
5.1 Initialization steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249
5.2 Execution frequency . . . . . . . . . . . . . . . . . . . . . . . . . . . 250
5.3 Known datasources . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
5.4 Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
5.5 Creating a customized image . . . . . . . . . . . . . . . . . . . . . . 252
5.6 Cloud-init in Windows . . . . . . . . . . . . . . . . . . . . . . . . . . 255
5.7 Troubleshooting cloud-init . . . . . . . . . . . . . . . . . . . . . . . . 255
5.8 Force password change during first login . . . . . . . . . . . . . . . 255
5.9 Self incrementing disk size during boot . . . . . . . . . . . . . . . . 256
5.10 Interactions with user-data . . . . . . . . . . . . . . . . . . . . . . . 257
6 Conclusion 258
Appendix A Terminology 260
9
List of Figures
1 Cloud computing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2 Home dashboard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
3 Relation diagram between accounts, domains, projects and users. 11
4 Browsing to the page for adding accounts . . . . . . . . . . . . . . 13
5 Adding a new account . . . . . . . . . . . . . . . . . . . . . . . . . . 13
6 Browsing to the account details page . . . . . . . . . . . . . . . . . 14
7 Checking account details . . . . . . . . . . . . . . . . . . . . . . . . . 14
8 Removing an account . . . . . . . . . . . . . . . . . . . . . . . . . . 15
9 Confirming the account removal . . . . . . . . . . . . . . . . . . . . 15
10 Editing limits for an account . . . . . . . . . . . . . . . . . . . . . . . 16
11 Editing the account settings . . . . . . . . . . . . . . . . . . . . . . . 17
12 Disabling an account . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
13 Confirming to disable the account . . . . . . . . . . . . . . . . . . . 18
14 Blocking an account . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
15 Confirming an account block . . . . . . . . . . . . . . . . . . . . . . 18
16 Defining two IPv4 ranges and one IPv6 range . . . . . . . . . . . . 19
17 Starting to create a new role . . . . . . . . . . . . . . . . . . . . . . 20
18 Creating the new role . . . . . . . . . . . . . . . . . . . . . . . . . . 21
19 Accessing the newly created role . . . . . . . . . . . . . . . . . . . . 22
20 Customizing the role’s rules . . . . . . . . . . . . . . . . . . . . . . . 22
21 Starting to edit a role . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
22 Editing the role . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
23 Removing a role . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
24 Confirming the role removal . . . . . . . . . . . . . . . . . . . . . . 23
25 Accessing the domains page . . . . . . . . . . . . . . . . . . . . . . 25
26 Creating a new domain . . . . . . . . . . . . . . . . . . . . . . . . . 25
27 Defining the new domain’s name . . . . . . . . . . . . . . . . . . . . 26
10
28 Visualizing the created domain . . . . . . . . . . . . . . . . . . . . . 26
29 Selecting the created domain . . . . . . . . . . . . . . . . . . . . . . 26
30 Starting to edit the new domain . . . . . . . . . . . . . . . . . . . . 27
31 Editing the domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
32 Browsing to the accounts page . . . . . . . . . . . . . . . . . . . . . 30
33 Select the account that the new user will be part of . . . . . . . . . 30
34 Check the users from the account . . . . . . . . . . . . . . . . . . . 31
35 Adding a new user . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
36 Define the new user informations and save changes . . . . . . . . 31
37 Selecting the user to edit it . . . . . . . . . . . . . . . . . . . . . . . 32
38 Starting to edit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
39 Editing the user . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
40 Deleting an user . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
41 Confirm the user removal . . . . . . . . . . . . . . . . . . . . . . . . 33
42 Disabling an user . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
43 Selecting the user to be enabled again . . . . . . . . . . . . . . . . 34
44 Enabling an user . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
45 Finding these settings . . . . . . . . . . . . . . . . . . . . . . . . . . 35
46 Message displayed when trying to log in with a disabled user . . . 36
47 Generating access keys for an user . . . . . . . . . . . . . . . . . . 37
48 Confirmation message when generating access keys . . . . . . . . 37
49 Configuring the two factor authentication . . . . . . . . . . . . . . 39
50 Two factor authentication provider options. . . . . . . . . . . . . . 39
51 Configuring the two factor authentication with TOTP. . . . . . . . . 40
52 Verification with TOTP during login. . . . . . . . . . . . . . . . . . . 40
53 Configuring the two factor authentication with static PIN. . . . . . 41
54 Verification using static PIN duing login. . . . . . . . . . . . . . . . . 41
55 Deactivating the two factor authentication. . . . . . . . . . . . . . . 42
56 Configuring the two factor authentication on login. . . . . . . . . . 42
11
57 Example of how the time zone affects the Apache CloudStack in-
terface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
58 Editing the user’s timezone . . . . . . . . . . . . . . . . . . . . . . . 43
59 Activating local timezone functionality . . . . . . . . . . . . . . . . . 44
60 Starting to create a new project . . . . . . . . . . . . . . . . . . . . . 45
61 Creating a new project . . . . . . . . . . . . . . . . . . . . . . . . . . 45
62 Resource limits for a project . . . . . . . . . . . . . . . . . . . . . . . 46
63 Changing resource limits from a project . . . . . . . . . . . . . . . . 47
64 Starting to create a project role . . . . . . . . . . . . . . . . . . . . . 48
65 Creating a project role . . . . . . . . . . . . . . . . . . . . . . . . . . 48
66 Defining the project role rules . . . . . . . . . . . . . . . . . . . . . 48
67 Starting to add members to the project . . . . . . . . . . . . . . . . 49
68 Adding an account to the project . . . . . . . . . . . . . . . . . . . . 49
69 Adding an user to the project . . . . . . . . . . . . . . . . . . . . . . 50
70 Selecting the view . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
71 Starting to edit a project . . . . . . . . . . . . . . . . . . . . . . . . . 52
72 Editing the project . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
73 Starting to suspend a project . . . . . . . . . . . . . . . . . . . . . . 53
74 Suspending a project . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
75 Starting the project removal . . . . . . . . . . . . . . . . . . . . . . . 54
76 Removing the project . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
77 Adding a VM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
78 Creating a VM with a user account of root admin type . . . . . . . 57
79 Creating a VM as an admin . . . . . . . . . . . . . . . . . . . . . . . 59
80 Creating a VM as an user . . . . . . . . . . . . . . . . . . . . . . . . 61
81 Viewing initialization options . . . . . . . . . . . . . . . . . . . . . . 63
82 Initialization options in a VM deploy form . . . . . . . . . . . . . . . 63
83 VM’s final settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
84 Choosing the VM to destroy . . . . . . . . . . . . . . . . . . . . . . . 65
12
85 Confirming VM destruction . . . . . . . . . . . . . . . . . . . . . . . 66
86 Choosing the VM to be destroyed . . . . . . . . . . . . . . . . . . . 66
87 Confirming VM destruction . . . . . . . . . . . . . . . . . . . . . . . 66
88 Stopping and starting a VM . . . . . . . . . . . . . . . . . . . . . . . 67
89 Choosing the VM to restart . . . . . . . . . . . . . . . . . . . . . . . 67
90 Restarting the VM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
91 Buttons for a VM console access and URL copying . . . . . . . . . 68
92 Return from the createConsoleEndpoint API . . . . . . . . . . . . . 68
93 Enabling dynamic scaling in a VM . . . . . . . . . . . . . . . . . . . . 69
94 Configuring dynamic scaling in a template . . . . . . . . . . . . . . . 70
95 Choosing the VM that will have its account migrated . . . . . . . . 72
96 Migrating the VM’s account . . . . . . . . . . . . . . . . . . . . . . . 73
97 Adding user-data when creating the VM . . . . . . . . . . . . . . . . 74
98 Changing user-data when editing the VM . . . . . . . . . . . . . . . 75
99 Verifying the user-data functionality . . . . . . . . . . . . . . . . . . 75
100 VM settings tab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
101 Visualizing all the volumes . . . . . . . . . . . . . . . . . . . . . . . . 79
102 Creating a new volume . . . . . . . . . . . . . . . . . . . . . . . . . . 80
103 Configuring the new volume . . . . . . . . . . . . . . . . . . . . . . 80
104 Verifying the new volume . . . . . . . . . . . . . . . . . . . . . . . . 80
105 Uploading a local volume . . . . . . . . . . . . . . . . . . . . . . . . 81
106 Performing the upload . . . . . . . . . . . . . . . . . . . . . . . . . . 81
107 Uploading a volume . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
108 Configuring the volume upload . . . . . . . . . . . . . . . . . . . . . 83
109 Accessing the volume details . . . . . . . . . . . . . . . . . . . . . . 84
110 Adding the volume to a VM . . . . . . . . . . . . . . . . . . . . . . . 84
111 Configuring the VM that will receive the new volume . . . . . . . . 85
112 Removing the volume from a VM . . . . . . . . . . . . . . . . . . . . 85
113 Confirming the operation . . . . . . . . . . . . . . . . . . . . . . . . 85
13
114 Resizing a volume . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
115 Configuring the resizing . . . . . . . . . . . . . . . . . . . . . . . . . 87
116 Downloading a volume . . . . . . . . . . . . . . . . . . . . . . . . . . 87
117 Confirming the download . . . . . . . . . . . . . . . . . . . . . . . . 87
118 Automating for volume snapshots . . . . . . . . . . . . . . . . . . . 88
119 Configuring the snapshot automation . . . . . . . . . . . . . . . . . 88
120 Removing a volume . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
121 Confirming the operation . . . . . . . . . . . . . . . . . . . . . . . . 89
122 Browsing to the guest network . . . . . . . . . . . . . . . . . . . . . 90
123 Rede isolada . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
124 Created isolated network . . . . . . . . . . . . . . . . . . . . . . . . 91
125 Creating a L2 network . . . . . . . . . . . . . . . . . . . . . . . . . . 92
126 Created L2 network . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
127 Rede shared . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
128 Created shared network . . . . . . . . . . . . . . . . . . . . . . . . . 95
129 Accessing the details for a guest network . . . . . . . . . . . . . . . 96
130 Removing a network . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
131 Confirming the network removal . . . . . . . . . . . . . . . . . . . . 96
132 Editing a network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
133 Restarting a network . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
134 Configuring the egress rules . . . . . . . . . . . . . . . . . . . . . . 98
135 Accessing theclient-network network. . . . . . . . . . . . . . . . . . 99
136 Adding an account with Add network permission. . . . . . . . . . . 100
137 After adding the network permission. . . . . . . . . . . . . . . . . . 101
138 Accessing the public IP . . . . . . . . . . . . . . . . . . . . . . . . . . 102
139 Viewing the public IP details . . . . . . . . . . . . . . . . . . . . . . . 102
140 Setting up the firewall . . . . . . . . . . . . . . . . . . . . . . . . . . 103
141 Enabling static NAT . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
142 Configuring the IP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
14
143 Verifying the final settings . . . . . . . . . . . . . . . . . . . . . . . . 104
144 Accessing and setting up port forwarding in the ports tab . . . . . 105
145 Choosing the VM that will have a port mapped . . . . . . . . . . . 105
146 Example for port forwarding setup with several VMs . . . . . . . . 106
147 Accessing and setting up the ports for load balancing . . . . . . . 106
148 Adding the VMs that will be managed by the load balancer . . . . 107
149 Created load balancer . . . . . . . . . . . . . . . . . . . . . . . . . . 107
150 Removing the public IP . . . . . . . . . . . . . . . . . . . . . . . . . . 108
151 Accessing the network details . . . . . . . . . . . . . . . . . . . . . . 108
152 Access the network public IP . . . . . . . . . . . . . . . . . . . . . . 109
153 Acquire a new IP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
154 List of available IPs . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
155 New IP acquired . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
156 Criando uma nova VPC . . . . . . . . . . . . . . . . . . . . . . . . . . 111
157 Configuring a new VPC . . . . . . . . . . . . . . . . . . . . . . . . . . 111
158 Verifying if the VPC were correctly created . . . . . . . . . . . . . . 112
159 Restarting a VPC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
160 Options for VPC restart . . . . . . . . . . . . . . . . . . . . . . . . . 113
161 Removing a VPC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
162 Confirming the operation . . . . . . . . . . . . . . . . . . . . . . . . 114
163 Possible error while trying to remove a VPC . . . . . . . . . . . . . 114
164 Accessing the VPC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
165 Accessing the public IPs of the VPC . . . . . . . . . . . . . . . . . . . 115
166 Choosing a public IP . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
167 Accessing the created VPC . . . . . . . . . . . . . . . . . . . . . . . . 116
168 Verifying the VPC details . . . . . . . . . . . . . . . . . . . . . . . . . 116
169 Adding a new tier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
170 Creating a new tier . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
171 Browsing to tiers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
15
172 Removing a tier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
173 Confirming the tier removal . . . . . . . . . . . . . . . . . . . . . . . 118
174 Adding a new ACL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
175 Creating a new ACL . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
176 Browsing to the created ACL . . . . . . . . . . . . . . . . . . . . . . 120
177 Browsing to new ACL rule creation . . . . . . . . . . . . . . . . . . . 120
178 Adding rule to allow traffic coming in through the port 22 from
the TCP protocol (SSH) . . . . . . . . . . . . . . . . . . . . . . . . . . 121
179 Adding rule to allow traffic going out through the port 22 from
the TCP protocol (SSH) . . . . . . . . . . . . . . . . . . . . . . . . . . 122
180 Selecting an account to own the tier . . . . . . . . . . . . . . . . . . 123
181 Selecting the account in the public IP acquisition form . . . . . . . 124
182 Selecting the VPC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
183 VPC details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
184 Public IP details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
185 Habilitando a VPN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
186 VPN enabled . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
187 Desabling a VPN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
188 Managing VPN users . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
189 Starting to add a VPN user . . . . . . . . . . . . . . . . . . . . . . . . 128
190 Adding a VPN user . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
191 Removing a VPN user . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
192 Starting to add a VPN on Linux . . . . . . . . . . . . . . . . . . . . . 130
193 Adding a VPN on Linux . . . . . . . . . . . . . . . . . . . . . . . . . . 130
194 Informing the IPSec pre-shared key . . . . . . . . . . . . . . . . . . 131
195 Informing the IPSec pre-shared key . . . . . . . . . . . . . . . . . . 131
196 Disabling the EAP protocol . . . . . . . . . . . . . . . . . . . . . . . 131
197 Disabling the EAP protocol . . . . . . . . . . . . . . . . . . . . . . . 132
198 Adding a VPN on Windows . . . . . . . . . . . . . . . . . . . . . . . 132
16
199 Starting to add a VPN on Windows . . . . . . . . . . . . . . . . . . . 133
200 Finishing to add a VPN on Windows . . . . . . . . . . . . . . . . . . 133
201 Changing the PPP settings . . . . . . . . . . . . . . . . . . . . . . . . 134
202 Enabling the MS-CHAP v2 protocol . . . . . . . . . . . . . . . . . . . 135
203 Disabling IPv6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
204 Disabling the use of default gateway in the remote network . . . . 136
205 Browsing to the VPN customer gateway tab . . . . . . . . . . . . . 136
206 Creating a VPN customer gateway . . . . . . . . . . . . . . . . . . . 137
207 Editing a VPN customer gateway . . . . . . . . . . . . . . . . . . . . 138
208 Deleting a VPN customer gateway . . . . . . . . . . . . . . . . . . . 138
209 Browsing to the VPN gateway under the VPC details . . . . . . . . 138
210 Creating a VPN gateway . . . . . . . . . . . . . . . . . . . . . . . . . 139
211 Browsing to the VPN connection under the VPC details . . . . . . . 139
212 Creating a VPN S2S connection . . . . . . . . . . . . . . . . . . . . . 140
213 Restarting a VPN S2S connection . . . . . . . . . . . . . . . . . . . . 140
214 Removing a VPN S2S connection . . . . . . . . . . . . . . . . . . . . 140
215 Generating keys. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
216 Viewing templates and ISOs . . . . . . . . . . . . . . . . . . . . . . . 148
217 Accessing the template details . . . . . . . . . . . . . . . . . . . . . 148
218 Template details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
219 Accessing the ISO details . . . . . . . . . . . . . . . . . . . . . . . . . 149
220 ISO details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
221 Listing filters for ISOs and templates. . . . . . . . . . . . . . . . . . 151
222 Registering the template . . . . . . . . . . . . . . . . . . . . . . . . . 152
223 Configuring the template . . . . . . . . . . . . . . . . . . . . . . . . 153
224 Registering the ISO . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
225 Configuring the ISO . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
226 Uploading the template . . . . . . . . . . . . . . . . . . . . . . . . . 155
227 Configuring the template . . . . . . . . . . . . . . . . . . . . . . . . 155
17
228 Uploading the ISO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
229 Configuring the ISO . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
230 Editing the template details . . . . . . . . . . . . . . . . . . . . . . . 157
231 Editing the template . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
232 Editing the ISO details . . . . . . . . . . . . . . . . . . . . . . . . . . 158
233 Editing the ISO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
234 Editing the template permissions . . . . . . . . . . . . . . . . . . . . 159
235 Editing the ISO permissions . . . . . . . . . . . . . . . . . . . . . . . 159
236 Editing permissions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
237 Editing the template sharing . . . . . . . . . . . . . . . . . . . . . . 160
238 Editing the ISO sharing . . . . . . . . . . . . . . . . . . . . . . . . . . 160
239 Editing the sharing options . . . . . . . . . . . . . . . . . . . . . . . 160
240 Deleting a template . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
241 Deleting an ISO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
242 Browsing to the VMs . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
243 Selecting a volume from the VM . . . . . . . . . . . . . . . . . . . . 163
244 Creating a template . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
245 Configuring a template . . . . . . . . . . . . . . . . . . . . . . . . . . 164
246 Resource tags from a given component . . . . . . . . . . . . . . . . 165
247 Browsing to the affinity groups . . . . . . . . . . . . . . . . . . . . . 166
248 Creating an affinity group . . . . . . . . . . . . . . . . . . . . . . . . 167
249 Affinity group created . . . . . . . . . . . . . . . . . . . . . . . . . . 167
250 Accessing the VM that will be moved to the group . . . . . . . . . . 167
251 Moving a VM to an affinity group . . . . . . . . . . . . . . . . . . . . 168
252 Selecting the group . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
253 Accessing the group . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
254 Viewing the VMs in the group . . . . . . . . . . . . . . . . . . . . . . 169
255 List of VMs in the group . . . . . . . . . . . . . . . . . . . . . . . . . 169
256 Removing the group . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
18
257 Creating an isolated network . . . . . . . . . . . . . . . . . . . . . . 171
258 Accessing the Public IP addresses tab . . . . . . . . . . . . . . . . . . 172
259 Acquiring a new public IP . . . . . . . . . . . . . . . . . . . . . . . . 172
260 Adding a load balancing rule . . . . . . . . . . . . . . . . . . . . . . 173
261 Selecting the isolated network . . . . . . . . . . . . . . . . . . . . . 173
262 Selecting the load balancing rule . . . . . . . . . . . . . . . . . . . . 174
263 Setting up the scale-up policy . . . . . . . . . . . . . . . . . . . . . . 175
264 Setting up a scale-down policy . . . . . . . . . . . . . . . . . . . . . 176
265 Configuring the autoscale VM group details . . . . . . . . . . . . . 177
266 Visualizing the instances from a autoscale VM group . . . . . . . . 177
267 Instances from the autoscale VM group . . . . . . . . . . . . . . . . 178
268 Instances in the autoscale VM group after the scale-up . . . . . . . 178
269 Instances in the autoscale VM group after the scale-down . . . . . 179
270 Deactivating the autoscale VM group . . . . . . . . . . . . . . . . . 179
271 Confirming the operation . . . . . . . . . . . . . . . . . . . . . . . . 179
272 Editing group details and VM deploy parameters . . . . . . . . . . 180
273 Dialog screen for editing group details and VM deploy parameters 180
274 Editing the group scale-up policy . . . . . . . . . . . . . . . . . . . . 181
275 Editing the group scale-down policy . . . . . . . . . . . . . . . . . . 181
276 Activating the autoscale VM group . . . . . . . . . . . . . . . . . . . 182
277 Confirming the operation . . . . . . . . . . . . . . . . . . . . . . . . 182
278 Removing the autoscale VM group . . . . . . . . . . . . . . . . . . . 182
279 Confirming the operation . . . . . . . . . . . . . . . . . . . . . . . . 183
280 Form for adding a Kubernetes version . . . . . . . . . . . . . . . . 185
281 Deleting Kubernetes ISOs via UI . . . . . . . . . . . . . . . . . . . . 186
282 Confirming the removal of a Kubernetes ISO . . . . . . . . . . . . . 186
283 Form for creating a Kubernetes cluster . . . . . . . . . . . . . . . . 188
284 Form for scaling a Kubernetes cluster . . . . . . . . . . . . . . . . . 189
285 Form for upgrading a Kubernetes cluster . . . . . . . . . . . . . . . 190
19
286 Accessing the Kubernetes . . . . . . . . . . . . . . . . . . . . . . . . 191
287 Accessing the VM details . . . . . . . . . . . . . . . . . . . . . . . . . 194
288 Taking a snapshot of the volume . . . . . . . . . . . . . . . . . . . . 194
289 Choosing which volume to take a snapshot . . . . . . . . . . . . . . 194
290 Configuring the snapshot . . . . . . . . . . . . . . . . . . . . . . . . 195
291 Taking a snapshot of the VM . . . . . . . . . . . . . . . . . . . . . . 195
292 Configuring the snapshot . . . . . . . . . . . . . . . . . . . . . . . . 195
293 Accessing the snapshots . . . . . . . . . . . . . . . . . . . . . . . . . 196
294 Accessing the volume snapshots . . . . . . . . . . . . . . . . . . . . 196
295 Accessing the VM snapshots . . . . . . . . . . . . . . . . . . . . . . 196
296 Accessing the snapshot details . . . . . . . . . . . . . . . . . . . . . 197
297 Creating a template . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
298 Configuring the template information . . . . . . . . . . . . . . . . . 198
299 Creating a new volume . . . . . . . . . . . . . . . . . . . . . . . . . . 198
300 Configuring the new volume . . . . . . . . . . . . . . . . . . . . . . 199
301 Reverting to the snapshot . . . . . . . . . . . . . . . . . . . . . . . . 199
302 Confirming the operation . . . . . . . . . . . . . . . . . . . . . . . . 199
303 Removing a volume snapshot . . . . . . . . . . . . . . . . . . . . . . 200
304 Confirming the removal . . . . . . . . . . . . . . . . . . . . . . . . . 200
305 Accessing the VM snapshot details . . . . . . . . . . . . . . . . . . . 200
306 Creating a volume snapshot from a VM snapshot . . . . . . . . . . 201
307 Configuring which volume will be extracted . . . . . . . . . . . . . 201
308 Creating a volume snapshot . . . . . . . . . . . . . . . . . . . . . . . 201
309 Reverting the snapshot . . . . . . . . . . . . . . . . . . . . . . . . . 202
310 Confirming the operation . . . . . . . . . . . . . . . . . . . . . . . . 202
311 Removing a VM snapshot . . . . . . . . . . . . . . . . . . . . . . . . 202
312 Confirming the operation . . . . . . . . . . . . . . . . . . . . . . . . 203
313 Browsing through the events . . . . . . . . . . . . . . . . . . . . . . 204
314 Archiving or removing an event . . . . . . . . . . . . . . . . . . . . . 204
20
315 Enabling the setting . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
316 Example for pop-ups from help icons . . . . . . . . . . . . . . . . . 206
317 Starting to create a new disk offering . . . . . . . . . . . . . . . . . 207
318 Creating a new disk offering . . . . . . . . . . . . . . . . . . . . . . . 208
319 Starting to edit the disk offering . . . . . . . . . . . . . . . . . . . . 210
320 Editing the disk offering . . . . . . . . . . . . . . . . . . . . . . . . . 211
321 Starting to update the disk offering access . . . . . . . . . . . . . . 211
322 Updating the disk offering access . . . . . . . . . . . . . . . . . . . 211
323 Changing the disk offering order . . . . . . . . . . . . . . . . . . . . 212
324 Starting to remove the disk offering . . . . . . . . . . . . . . . . . . 212
325 Removing the disk offering . . . . . . . . . . . . . . . . . . . . . . . 212
326 Starting to create a new compute offering . . . . . . . . . . . . . . 214
327 Creating a new compute offering - 1 - continues . . . . . . . . . . . 215
328 Starting to edit the compute offering . . . . . . . . . . . . . . . . . 218
329 Editing the compute offering . . . . . . . . . . . . . . . . . . . . . . 218
330 Starting to update the compute offering access . . . . . . . . . . . 219
331 Updating the compute offering access . . . . . . . . . . . . . . . . 219
332 Changing the compute offering order . . . . . . . . . . . . . . . . . 219
333 Starting the compute offering removal . . . . . . . . . . . . . . . . 219
334 Removing the compute offering . . . . . . . . . . . . . . . . . . . . 220
335 Starting to change the compute offering . . . . . . . . . . . . . . . 220
336 Changing the compute offering . . . . . . . . . . . . . . . . . . . . . 221
337 Changing the compute offering . . . . . . . . . . . . . . . . . . . . . 221
338 Starting to create a network offering . . . . . . . . . . . . . . . . . . 223
339 Creating a new network offering - 1 - continues . . . . . . . . . . . 223
340 Creating a new network offering - 2 . . . . . . . . . . . . . . . . . . 225
341 Starting to edit the network offering . . . . . . . . . . . . . . . . . . 226
342 Editing the network offering . . . . . . . . . . . . . . . . . . . . . . . 226
343 Starting to disable the network offering . . . . . . . . . . . . . . . . 227
21
344 Disabling the network offering . . . . . . . . . . . . . . . . . . . . . 227
345 Starting to update the network offering access . . . . . . . . . . . 227
346 Updating the network offering access . . . . . . . . . . . . . . . . . 227
347 Changing the network offering order . . . . . . . . . . . . . . . . . 228
348 Starting the network offering removal . . . . . . . . . . . . . . . . . 228
349 Removing the network offering . . . . . . . . . . . . . . . . . . . . . 228
350 Starting to create a new VPC offering . . . . . . . . . . . . . . . . . 229
351 Creating a new VPC offering . . . . . . . . . . . . . . . . . . . . . . . 230
352 Starting to edit a VPC offering . . . . . . . . . . . . . . . . . . . . . . 231
353 Editing a VPC offering . . . . . . . . . . . . . . . . . . . . . . . . . . 231
354 Starting to disable the VPC offering . . . . . . . . . . . . . . . . . . 231
355 D disabling the VPC offering . . . . . . . . . . . . . . . . . . . . . . . 231
356 Starting to update the VPC offering access . . . . . . . . . . . . . . 232
357 Updating the VPC offering access . . . . . . . . . . . . . . . . . . . 232
358 Changing the VPC offering order . . . . . . . . . . . . . . . . . . . . 232
359 Starting the VPC offering removal . . . . . . . . . . . . . . . . . . . 232
360 Removing the VPC offering . . . . . . . . . . . . . . . . . . . . . . . 233
361 Utilizando o autocomplete . . . . . . . . . . . . . . . . . . . . . . . . 236
362 Checking the autocomplete for a specific command . . . . . . . . 236
363 Checking the autocomplete for a specific API . . . . . . . . . . . . . 236
364 Using the -h flag to get help . . . . . . . . . . . . . . . . . . . . . . . 237
365 Performing the API call . . . . . . . . . . . . . . . . . . . . . . . . . . 238
366 Deploying a VM via CloudMonkey . . . . . . . . . . . . . . . . . . . 239
367 Active tariffs listing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241
368 Tariffs details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241
369 Accessing details and reports from an account . . . . . . . . . . . 242
370 Credits listing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242
371 Resource usage listing . . . . . . . . . . . . . . . . . . . . . . . . . . 244
372 Detailed listing for resource usage . . . . . . . . . . . . . . . . . . 245
22
List of Tables
24
Notes
• This document don’t cover all the topics, subjects and use cases for Apache
CloudStack. Therefore, in case you don’t find what you’re looking for, do
a review or inclusion solicitation through GitLab.
• This document is periodically updated. So, stay tuned for new updates.
25
1. Introduction
During the last few years, there was a huge growth in the usage of cloud
computing (private, public and/or hybrid), intensifying opportunities in the dig-
ital services market. For supplying this demand the infrastructure needed to
created and maintain this kind of environment grows constantly, directly im-
pacting the management and operation costs.
Cloud computing environments are complex and heterogeneous, possess-
ing dynamic variables and many components that must be interwined and man-
aged. For an efficient management of this kind of environment it’s necessary
the usage of tools capable of orchestrating the infrastructure, helping during
implementation, maintenance and management of cloud services and systems.
Apache CloudStack and OpenStack are the main open-source alternatives
for creating private cloud environments, highlighted by their robustness, project
maturity and range of technologies and functionalities supported. Such charac-
teristics translate in hundreds of companies that adopt such platforms around
the world, including companies like: USP, UNICAMP, Dell, Deutsche Telekom,
Apple, Planetel, Locaweb and many others.
SC Clouds comes in aid of this need, providing consultancy, support and con-
tinued development to cloud infrastructure providers and companies that have
the cloud technology as the main pillar for creating and providing services. In
partnership with clients is carried out the planning and execution for the cloud
computing environments that provide infrastructure as a service. Our profes-
sionals have full mastery with the Apache CloudStack and OpenStack platforms,
being responsible for various new features and bug fixes in the projects. Cur-
rently, SC Clouds helps with management, planning, implantation, support, bug
fixing and implementing new functionalities for companies in Germany, Brazil,
Italy, Mexico and Swiss.
26
1.1. Private cloud platforms
Cloud orchestration platforms, such as Apache CloudStack and OpenStack,
are used to provide private, public and hybrid cloud in companies and institu-
tions around the globe. They are capable to connect different computational
systems (storage, network and virtualizers), creating the abstraction for infrastructure-
as-a-service for the system users and administrators.
Both OpenStack and CloudStack are free open-source softwares for cloud
environment orchestration. Both share the goal of managing computational
resources, working with the virtualization concept to provide resources on de-
mand, focusing in the provision infrastructure-as-a-service resources.
Figure 1: Cloud computing
1.1.1. Basic functionalities
Both CloudStack and OpenStack offer basic functionalities needed for pro-
vision of cloud computing services. Some of the provided functionalities are:
• Computing services
– VM provision and management fo their resources (networks, mem-
ory, CPU, disks, etc);
27
– Manage the images which the VMs will be created
– Support for multiple hypervisors, like KVM, XenServer and VMware.
• Network services
– Management and configuration for shared and dedicated networks
– VLAN, VxLANS (overlay networks) and other topologies creation
– Define access and routing policies
– Qos, Load balancers and firewalls management;
• Storage services
– Disks management
– Block and object based storage
– Backups;
– SDS support (Software Defined Storage), e.g. Ceph/RBD.
• Resource monitoring services
– event monitoring;
– Display for available and used resources
– Pricing and billing.
• Open REST APIs, standardized and documented, to manage all services,
their automation and integration with other tools;
• Authentication service with native LDAP integration;
• Support to SAML2 and OpenID Connect federated systems;
• Containers-as-a-service.
Within basic functionalities for cloud orchestration system, Apache Cloud-
Stack have limitations compared to federated systems, since it supports only
the SAML2 protocol.
28
1.2. Apache CloudStack history
The CloudStack project started in 2008 as a startup project known as VMOps.
The company changed its name to Cloud.com and released a CloudStack stable
version in 2010, under the GNU GPLv3 license (General Public License version
3). In 2011, Cloud.com was acquired by Citrix and in 2012 the project was do-
nated to the Apache foundation. In the last few years, CloudStack has grew up
and has become reference as a orchestration tool, being used by many organi-
zations, such as Globo, RNP, UNICAMP, USP, Locaweb, CGEE, Apple, Dell, Nokia
e Disney.
29
2. Apache CloudStack functionalities
In this section the main Apache CloudStack functionalities will be briefly in-
troduced as well as how to use them.
2.1. Home dashboard
This dashboard is the default in Apache CloudStack, where is possible to
visualize the amount of resources consumed by the user.
Figure 2: Home dashboard
2.2. Managing access to the cloud
CloudStack allows that the cloud administrator define and manage with pre-
cision which resources are available and for whom, through domains, accounts,
users, projects and roles. For such, firstly it’s important to understand these
concepts.
• Account: an account represents a client fo the cloud service provider
client, accessed or not by different users. In this context an account de-
fines, for example, an company department or a college within an univer-
sity, grouping the users from within these sectors.
30
• User: The user, in turn, is a member from the account. It belongs only
to one account and have access to resources based on the account type,
receiving the same privileges as the account that holds it. They are an alias
for the account.
• Domain: a domain may contain several accounts that, usually, have a
logical relation between themselves. They are used to limit the resources
that will be given to the users within this domain.
• Project: project are means to organize accounts and resources. The users
may be grouped in projects to share resources such as VMs, snapshots,
templates and IP addresses. Resources created in a project belong only
to it, not to an account, and can only by used within it. Members from
the project can visualize and manage the virtual resources created by any
account within the project.
• Roles: defined at account and/or project level, they are sets that define
which permission that accounts and users within the environment will
have, such as creating, listing and removing resources.
The following image shows a relation diagram between accounts, domains,
projects and users.
Figure 3: Relation diagram between accounts, domains, projects and users.
31
2.2.1. Account
Represents a client that uses the cloud services. In CloudStack, an account
may be used for more than one user, to the point that a whole organization
may use the same account. It can be divided in three basic types for visualizing
resources, with those types being defined when creating the role:
• Admin: has access to all of the environment resources, including infras-
tructure elements and platform settings. These accounts can also be called
as root admin.
• DomainAdmin: has access to all elements allocated within its domain,
including user management.
• ResourceAdmin: has access to all elements allocated within its domain,
except for user management.
• User: has access only to elements allocated in its account.
The set of permissions for each account type is defined through roles; how-
ever, some actions, as well as the scope of the visualized resources, depends
on the account type.
An account can have its type changed after created, but it can have its per-
missions changed.
It’s possible to perform the following operations over accounts:
32
2.2.1.1. Adding an account
Figure 4: Browsing to the page for adding accounts
Figure 5: Adding a new account
33
2.2.1.2. Visualizing account details
Figure 6: Browsing to the account details page
Figure 7: Checking account details
All operations detailed below are performed in this page.
34
2.2.1.3. Removing an account
Figure 8: Removing an account
When attempting to delete an account, a pop-up will ask for confirmation for
this action. During the removal process, all users and resources (VMs, snap-
shots, templates, volumes, etc) related to the account will also be removed.
Figure 9: Confirming the account removal
2.2.1.4. Configuring account accesses
There are two wayt to setup an account’s accesses:
• Edit the account limits: this is the most basic way to configure the account
accesses, in which the system administrator defines resources limits that
the account may not surpass.
35
Figure 10: Editing limits for an account
• Editing the account settings: contain options more advanced to configure
its accesses.
36
Figure 11: Editing the account settings
2.2.1.5. Disabling an account
This option prevents that the account users from accessing and managing any
resource, while also automatically shuting down all VMs created by them. It can
be enabled again later.
Figure 12: Disabling an account
37
Figure 13: Confirming to disable the account
2.2.1.6. Blocking an account
Similar to the option of disabling an account, however it just prevents the user
to create and manage new resources. Already existing resourcer are not af-
fected.
Figure 14: Blocking an account
Figure 15: Confirming an account block
38
2.2.1.7. API access restriction by IP
Within the global settings, of account or domain
1
, it’s possible to change the
parameter api.allowed. source.cidr.list to define a list (comma separated) con-
taining the only IPv4 and IPv6 with permissions to access the CloudStack API.
The default value is 0.0.0.0/0,::/0, allowing the access through any IP.
If there’s any discrepancy between settings, account settings hold priority.
The global setting api.source.cidr.checks.enabled must be set to true to use
this function.
Figure 16: Defining two IPv4 ranges and one IPv6 range
The ACS uses the header X-Forwarded-For from the HTTP request to iden-
tify the source IP from the API requests. This allows to audit processes made
by the users and the IP restriction mentioned in this section. This header can
be changed by the requester, therefore it’s necessary to setup the firewalls and
proxies that intermediate the communicationwith ACS in a way that this is iden-
tified and that the header passed to ACS contains tje real source IP from the
requester.
2.2.2. Roles
Roles represents set of permission that accounts have within the cloud in-
frastructure. Beyond permission sets, as presented in Section ??, roles define
the account types, which indicate the scope for resource management. The
account types are:
• Admin: has access to all resources from the environment, including in-
frastructure elements and platform settings. These accounts are also called
1
If the global setting enable.account.settings.for.domain is true.
39
root admin.
• DomainAdmin: has access to all elements allocated within its domain,
including user management.
• ResourceAdmin: has access to all elements allocated within its domain,
except for user management.
• User: has access only to elements allocated in its account.
The CloudStack provides default roles, which cannot be edited or removed.
However, it’s possible to create custom roles, with complete access. The pro-
cedure to perform this action is the following:
2.2.2.1. Adding a new role
1. Browse to Roles and add a new role.
Figure 17: Starting to create a new role
2. Add the basic settings for the new role:
40
Figure 18: Creating the new role
The field Based on is mandatory. It can refer to already existing types or
roles. It’s used to determine which account type that a role may use. Ex-
ample: a role of Admin type can only be allocated to accounts of Admin
type.
About the two available options:
• Type: defines which account type can use this role.
• Role: uses an existing role to define which account type can use this
role and also copy the rules from contained in that role.
2.2.2.2. Visualizing role details
1. Browse to the details of the created role.
41
Figure 19: Accessing the newly created role
2.2.2.3. Customizing a role
1. Customize this role’s rules.
Figure 20: Customizing the role’s rules
To utilize this new role it’s necessary to create a new account using it.
2.2.2.4. Removing a role
1. It’s also possible to edit and remove roles, except for a situation that will
soon be covered.
42
Figure 21: Starting to edit a role
Figure 22: Editing the role
Figure 23: Removing a role
Figure 24: Confirming the role removal
43
Some important notes:
• Roles already associated with an account can’t be edited or removed, mak-
ing it necessary to remove the account first.
• A role may be used by many accounts, but an account can only use one
role.
• Besides roles associated with an account can’t be edited or deleted, it’s
still possible to change rules that compose that role, by editing, removing
or adding them.
2.2.3. Domains
Aggregation of several accounts, usually used to unify accounts with differ-
ent roles, but that are part of the same organization. Domains provides means
to isolate and limit access to cloud resources to accounts that compose it. The
standard CloudStack architecture already have a default domain, called ROOT,
which is "parent" to all other domains, while also having total access to the
cloud resources.
• The ROOT domain:
The ROOT domain is considered the default domain. It can’t be neither
deleted or have its cloud resources accesses reduced. Only users from
accounts using the root admin role can have access to this domain. From
this domain all the others are configured.
44
Figure 25: Accessing the domains page
2.2.3.1. Adding a domain
• When adding a new domain, it will be created as a subdomain (child) from
the current domain. In the example presented there’s only one domain.
However, within more complex organization, with multiple domains, it’s
necessary to pay attention to the hierarchy during the creation process.
Figure 26: Creating a new domain
45
Figure 27: Defining the new domain’s name
Figure 28: Visualizing the created domain
Figure 29: Selecting the created domain
2.2.3.2. Configuring the domain
• Configuring the new domain:
By default, new domains inherit from the parent domain the resource
access privileges However, these accesses may be changed.
46
Figure 30: Starting to edit the new domain
2.2.3.3. Defining the domain accesses
• Defining the accesses for the new domain:
By default, the value -1 indicates that there’s no limits, i.e., accounts in this
domain can use all the cloud resources. The value 0 indicates that they
can’t use that resource. Any value greater than 0 defines the maximum
amount that the accounts from that domain can utilize.
47
Figure 31: Editing the domain
With the new domain created and configured, the next step it’s to add ac-
counts to it. Currently, it’s not possible to migrate an account from a domain
to another. Therefore, for structures in which domains manages the cloud, it’s
necessary to create them before creating the accounts.
2.2.3.4. Restructuring the domain tree
To restructure the domain tree, the moveDomain can be used
2
:
move domain domainid=<domain_id> parentdomainid=<destination_parent_domain_id>
It’s important to highlight that for the domain to be moved, it’s necessary to
follow these restrictions:
• Both the domain and the parent domain must be active.
2
The parameter parentdomainid it’s optional. If not informed, the domain will be moved to
the parent domain ROOT.
48
• The new parent domain can’t be a subdomain from the domain itself.
• A domain with the same name as the domain being moved must not exist
in the new parent domain.
• The new parent domain must have access to all resources that the domain
and its subdomains utilize (shared networks and affinity groups).
2.2.4. Users
Represents an user that belongs to an account and have an unique user-
name within the domain. Users within the same domain can communicate with
each other and see other users actions, but can’t access users from others do-
mains. They have the privileges given to the account and act within a defined
domain. Depending on the account that the user is part of, it will have different
privileges within the domain. For example:
• Users without admin privileges can only use resources available inside the
domain, respecting the limitations from the account that it’s part of and
also limitations from the role from the accout used.
• Admin users can manage the resources, in addition to also manage ac-
counts and users within the domain.
It’s possible to add several users to the same account, each of them having a
its name and password. It’s also possible to edit or delete users already created.
However, these two operations are only allowed to a admin user.
2.2.4.1. Adding an user
The steps to add a new user are:
1. Go to Accounts.
49
Figure 32: Browsing to the accounts page
2. Select in which account the user will be added.
Figure 33: Select the account that the new user will be part of
3. Browse to the users page from this account.
50
Figure 34: Check the users from the account
4. Add a new user.
Figure 35: Adding a new user
Figure 36: Define the new user informations and save changes
2.2.4.2. Editing a user
To edit an user:
51
1. Repeat steps 1, 2 and 3 previously explained, but now select the account
which has the user to be edited.
2. Click in the user to be edited.
Figure 37: Selecting the user to edit it
3. Click in edit the user.
Figure 38: Starting to edit
4. Perform the desired changes and save.
Figure 39: Editing the user
2.2.4.3. Deleting an user
To delete an user:
52
1. Repeat all the steps (2) to get to the user details page.
2. Remove the user.
Figure 40: Deleting an user
Figure 41: Confirm the user removal
2.2.4.4. Disabling/Enabling an user
It’s also possible to disable an user, preventing it to access and manage any
resource.
Figure 42: Disabling an user
To give the access back it’s necessary that an administrator of the domain
to enable the disabled user.
53
Figure 43: Selecting the user to be enabled again
Figure 44: Enabling an user
2.2.4.5. Paswword policies
To improve the security of the cloud environment users, the operator can en-
able password policies (rules that the password must follow), which can be ac-
cessed in the global settings. It’s worth highlighting that these rules are only
imposed to user password and other passwords, such as those from VMs, are
not affected.
Currently there are 7 password policy options available:
• password.policy.allowPasswordToContainUsername: indicates if the pass-
word may contain the user name.
54
• password.policy.minimum.digits: minimum quantity of digits in the pass-
word.
• password.policy.minimum.length: minimum password length.
• password.policy.min imum .low erca se.l ette rs: minimum quantity of low-
ercase letters in the password.
• password.po licy.minimum.special.c haracters: minimum quantity of spe-
cial characters in the password.
• password.policy.minimum.uppercase.letters: minimum quantity of up-
percase letters in the password.
• password.policy.regex: regular expression that the password must follow.
Figure 45: Finding these settings
By default, the are no restrictions for password creation, all the settings
mentioned above are defined in the broadestly way possible. Is up to the oper-
ator to identify the need to implement these security measures and setup the
password policies accordingly.
55
2.2.4.6. Disabling the user for wrong password
The ACS has a functionality to disable the user for perfoming too many incorrect
login attempts. By default, it’s necessary that the user enter a wrong password
5 times in a row to be disabled. This value can be changed through the global
setting incorrect.login.attempts.allowed.
To enable it again, follow the steps described in section Disabling/Enabling
a user.
Figure 46: Message displayed when trying to log in with a disabled user
2.2.4.7. Managing user API keys
API keys are access credentials given as a way to authorize the use of specific
functionalities from one or more APIs. They often act as an unique identifier for
users. In CloudStack they may be used to perform operations and integrations
(often automatically), access usage data and specific CloudStack development,
test and integration tools.
The ACS allows that users generate their own keys and that administrators
create ad update access keys for their users. Some of the operations possible
for them are:
56
1. registerUserKeys: This command has as its input the user ID for which the
access keys will be generated, and as output shows the API key and the
secret key registered for the ID informed.
2. getUserKeys: Returns the registered keys for the user ID informed on the
input.
3. updateUser: In the context of access keys, the command updateUser al-
lows user keys to be updated, and for that, just inform the user ID and
both userapikey and usersecretkey parameters.
To generate keys through the UI, it’s needed to edit the user that the keys
will be generated for. Follow the steps described in Editing a user and click in
the button located in the upper right as shown in the image ??.
Figure 47: Generating access keys for an user
fig:generate-keys
After clicking in the button, a window requesting to confirm the action will
show up.
Figure 48: Confirmation message when generating access keys
Just confirm for the keys pair to be generated and registered for this user.
57
2.2.4.8. API keys usage in CloudMonkey
To utilize the access keys on CloudMonkey it’s possible to define them updat-
ing the settings file directly
3
. Just fill the apikey and secretkey parameters ade-
quately in the file and save it. The next time the CloudMonkey starts, it will use
the configured keys.
asyncblock = true
timeout = 1800
output = json
verifycert = true
profile = mylab
[mylab]
url = http://my-local-lab/client/api
domain = /
apikey = IsBXfV_Km9sgbc3zYsf6fiia2z19qCnB80S7quN2lSy0SvD97A
secretkey = crmgT-n820MGl8EYZ37yRvnjDjfQyV8FWHBb5EmFxN5JiqZg1ZHRk6vsNw
Another way to utilize the keys when using CloudMonkey it’s to manually
set the parameters. For that, open the CloudMonkey and execute the following
commands:
(mylab) > set username your-username
(mylab) > set domain your-domain
(mylab) > set apikey YOUR-API-KEY
(mylab) > set url https://your-lab/client/api
(mylab) > set secretkey YOUR-SECRET-KEY
After finishing these steps execute a sync command to verify if the changes
took effect as desired.
(mylab) > sync
Discovered 623 APIs
2.2.4.9. Two factors authentication
The CloudStack supports two factor authentication (2FA), in which the second
factor utilized can be the Google Authenticator, any authenticator with TOTP sup-
port or a static PIN.
This extra security layer can be setup for a domain of globally by the admin-
istrator through the following settings:
3
This file normally is gound in the folder home/.cmk/config or in
home/snap/cloudmonkey/current/.cmk/config depending the way it was installed.
58
Setting Description Value
Deafult
enable.user.2fa Determines if the two factor au-
thentication is enabled.
false
mandate.user.2fa Determines if the two factor au-
thentication is mandatory for the
users.
false
user.2fa.default.provider Default two factor authentication
provider.
totp
When enabling the two factor authentication, users may setup the 2FA ac-
cessing their profiles.
Figure 49: Configuring the two factor authentication
In the pop-up, the user must select one of the providers and perform the
respective configuration.
Figure 50: Two factor authentication provider options.
When choosing the Google Authenticator or some other authenticator that
supports the TOTP algorithm, it’s necessary to setup the account on the re-
59
spective application by scanning the QR code or using the configuration key
provided by CloudStack. After setting up, the user must provide the code gen-
erated by the application to perfom the login.
Figure 51: Configuring the two factor authentication with TOTP.
Figure 52: Verification with TOTP during login.
When choosing the static PIN, when the user is logging in, they will have to
provide the code generated by CloudStack during the 2FA setup.
60
Figure 53: Configuring the two factor authentication with static PIN.
Figure 54: Verification using static PIN duing login.
The user can deactivate the 2FA at any time in their profile
4
.
4
If the setting mandate.use r.2fa is true, the user will have to configure again the authenti-
cation during the next login.
61
Figure 55: Deactivating the two factor authentication.
Furthermore, the administrator can also make the two factor authentication
mandatory through the mandate.user.2fa setting. This way, users will have to
setup the authentication on their next login.
Figure 56: Configuring the two factor authentication on login.
If needed to make the 2FA mandatory for a single user, it’s possible to use
the API updateUser:
(mylab) > update user id=<user_id> mandate2fa=true
If an user lose the authentication application or forget the PIN, they will need
to contact the system administrator to disable the two factor authentication. It
can be deactivated accessing the user profile and selecting the option Disable
User Two Factor Authentication as previously explained.
62
If the administrator themself loses the two factor, they can call the API setu
pUserTwoFactorAuthentication using secretkey and apikey
5
:
(mylab) > setup usertwofactorauthentication userid=<admin_user_id> enable=false
If the administrator don’t have s ecretkey and apikey configured, it’s neces-
sary to contact the support for recovering the account.
2.2.4.10. Timezone
When creating a new user, it’s necessary to define a timezone for it (as in section
Adding a new user). This parameter is responsible for the times displayed in
various pages in the Apache CloudStack interface, making the time information
in accord with the chosen timezone.
Figure 57: Example of how the time zone affects the Apache CloudStack interface
It’s possible to edit the user’s current timezone. For that, follow the steps in
Editing an user.
Figure 58: Editing the user’s timezone
5
When using secretkey and apikey, the 2FA is ignored.
63
By default, the Apache CloudStack interface will show the time accordingly
to the chosen timezone. However, it’s possible to change the local timezone
without editing the user. For that, just activate the option "Use local timezone"
as shown in the image below:
Figure 59: Activating local timezone functionality
2.2.5. Projects
Projects are used to organize resources and group accounts and users. By
default, both admin users and regular users may create a project, with the cre-
ator automatically becoming its administrator. It’s possible to change this be-
haviour through the setting allow.user.create.projects:
Setting Description Value
Deafult
allow.user.create.projects Defines if a regular user can cre-
ate a project.
true
2.2.5.1. adding a project
For creating a project:
64
Figure 60: Starting to create a new project
When clicking in the button New Project, a form for filling up informations
will show up:
Figure 61: Creating a new project
Projects possess resource allocation limits:
65
Figure 62: Resource limits for a project
These limits are inherited from the global settings:
Setting Description Value
Deafult
max.project.cpus Maximum number of CPU cores
that a project can use.
40
max.project.memory Maximum amount of memory (in
MiB) that a project can use.
40960
max.project.networks Maximum number of networks
that can be created to a project.
20
max.project.primary.storage Maximum amount of primary
storage (in GiB) that a project can
use.
200
max.project.public.ips Maximum number of IPs that a
project can consume.
20
max.project.secondary.storage Maximum amount of secondary
storage (in GiB) that a project can
use.
400
66
Setting Description Value
Deafult
max.project.snapshots Maximum number of snapshots
that can be created to a project.
20
max.project.templates Maximum number of templates
that can be created to a project.
20
max.project.user.vms Maximum number os user VMs
that can be deployed for the
project.
20
max.project.volumes Maximum number of volumes
that can be created to a project.
20
max.project.vpcs Maximum number of VPCs that
can be created to a project.
20
However, they can be edited for each project:
Figure 63: Changing resource limits from a project
Projects also have roles, which are above the roles of an account. If an ac-
count is added to a project without a role, the account role will be used.
67
Figure 64: Starting to create a project role
Figure 65: Creating a project role
Figure 66: Defining the project role rules
2.2.5.2. Adding accounts and users to the project
To add accounts and users to the project, just fill the form and send an invite:
68
Figure 67: Starting to add members to the project
Figure 68: Adding an account to the project
69
Figure 69: Adding an user to the project
After finishing the addition, the account or user will be part of the project,
however it’s possible to change this behaviour so that it’s necessary for them
to confirm the invite, through an e-mail, before being effectively added to the
project. For that, there are the following settings:
Setting Description Value
Deafult
project.invite.required Defines if the account or user
need to accept the invite to the
project.
false
project.invite.timeout Time (in seconds) for invite expi-
ration.
86400
To make it possible to send the confirmation e-mails, it’s necessary to setup
the e-mail sending service for projects:
70
Setting Description Value
Deafult
project.email.sender Address in the header From from
the confirmation e-mail.
null
project.smtp.enabledSecurity
Protocols
Security protocols for send-
ing e-mails, separated with
spaces. Supported protocols:
SSLv2Hello; SSLv3; TLSv1;
TLSv1.1; TLSv1.2; TLSv1.3.
project.smtp.host SMTP server address for send-
ing e-mails.
null
project.smtp.password Password for authentication in
the SMTP server (only if the set-
ting project.smtp.useAuth is set to
true)
null/false
project.smtp.port Port which the SMTP is listening. 465
project.smtp.useAuth Defines if authentication will be
used to send e-mails.
null
project.smtp.username User for authentication on the
SMTP server (only if the set-
ting project.smtp.useAuth is set to
true)
null
project.smtp.useStartTLS Defines if StartTLS will be used
to protect the connection (only if
the setting project.smtp.useAuth
is set to true).
false
An account or user may be in multiples projects simultaneously. To be pos-
sible consume and visualize the project resources, it’s necessary to change the
visualization:
Figure 70: Selecting the view
71
Aside from the functionalities mentioned, it’s also possible to:
2.2.5.3. Editing a project
• Edit:
Figure 71: Starting to edit a project
Figure 72: Editing the project
2.2.5.4. Suspending a project
• Suspend:
72
Figure 73: Starting to suspend a project
Figure 74: Suspending a project
If a project has resources while it’s suspended, its VMs will be stopped and
its VRs may be destroyed.
2.2.5.5. Deleting a project
• E excluir:
73
Figure 75: Starting the project removal
Figure 76: Removing the project
If a project has resources wheh it’s removed, they will be destroyed.
More information in the official documentation.
2.2.6. Notes
There are situations that an user has a domain with multiple accounts, with
different roles, that will need to share resources. Based on the CloudStack ar-
chitecture, there are two alternatives for sharing resources between different
accounts:
74
• Through project, in which the accounts and users that will share resources
will be added. It’s also possible to create different project roles, individu-
ally limiting the access to shared resources.
• Creating a role for accounts, based on the DomainAdmin type. This role
will operate at the same level than the default DomainAdmin, however it
will be possible to limit access to the created DomainAdmin role through
access rules.
It’s important to notice that a project may only contain accounts from the
same domain that the project was created. It’s not possible to accounts from
different domains to be added to the same project.
2.3. VMs management
The following actions for VMs are available:
2.3.1. Adding a VM
Figure 77: Adding a VM
To add a VM, it’s necessary to specify some settings, which vary depending
with the account type of the user. It’s important to also pay attention to the
existence of components that fulfill the hardware requirements of that VM, for
not only CPU and RAM but also for network. To implant a VM based in a ISO,
the Apache CloudStack will use the data disk to perform the OS boot. With
that, the data disk will be changed to the root disk afterwards. Thus, only after
the VM is created that will be possible to actually allocate a data disk, since
75
the initial volume will become the root disk to perfom the operating system
implementation.
2.3.1.1. Adding a VM as root admin
76
Figure 78: Creating a VM with a user account of root admin type
77
2.3.1.2. Adding a VM as an admin
78
Figure 79: Creating a VM as an admin
79
2.3.1.3. Adding a VM as an user
80
Figure 80: Creating a VM as an user
2.3.1.4. Creating a VM with UEFI initialization
The UEFI (Unified Extensible Firmware Interface) allows to surpass the limitations
imposed by the BIOS. The difference between these two initialization types is
that UEFI stores all the data about the initialization in a .efi file instead of storing
in firmware. Here are some of the advantages of UEFI compared to the BIOS:
• Supports unit sizes up to 9 zetabytes (BIOS only up to 2,2 terabytes);
• Faster initialization time;
• Offers better security with "Secure Boot", preventing that the machine
boots from unauthorized applications;
• Runs in 32 or 64 bits (allowing using the mouse to browse), while BIOS
uses 16 bits (with only keyboard support for browsing)
81
The Libvirt conventionally uses the BIOS as default firmware to start the
guest VMs. For it to start VMs with UEFI, it’s necessary to install the ovmf pack-
ages, available on most Linux repositories, in the host where the VMs will be
implanted. Verify if the package is present in the host with the following com-
mand: dpkg -s ovmf. If the package is not found, do this:
• 1. For Debian based systems, execute:
sudo apt install ovmf
• 2. For RHEL based systems, it’s necessary to install the edk2-ovmf package
using the following command:
sudo dnf install edk2-ovmf
or
sudo yum install edk2-ovmf
• 3. For Arch Linux systems:
sudo pacman -S edk2-ovmf
For the second step it’s necessary to use a template in the VM that supports
UEFI initialization.
When implementing a VM, make sure that CloudStack considers for allo-
cation only hosts enabled for UEFI, as registered in the database. Setup the
virtual machine as needed, making adjustments to the XML definitions if using
the KVM hypervisor, to ensure UEFI compatibility.
82
Figure 81: Viewing initialization options
Figure 82: Initialization options in a VM deploy form
During VM creation, it’s possible to check the details of the final settings for
the VM before creating it:
83
Figure 83: VM’s final settings
84
2.3.2. Removing a VM
It’s position to perform VM removals through the CloudMonkey or via UI.
When perfoming such action, pay attention to the expunge parameter. By de-
fault, it’s set to false, allowing to recover a VM removed by a administrator. If
it’s set to true, the VM is completely removed and can’t be retrieved afterwards.
Through the API, it’s possible to use the destroyVirtu alMachine command
to put the VM in the Destroyed, state, marking it for permanent removal after
the period defined by the global setting expunge. d elay. The CloudStack make
purge checks based in the interval defined in the global setting expunge.interva
l. The API can be assisted by the expunge parameter. It’s worth to highlight that
roles of the Admin type will always be able to inform such parameter; accounts
with other role will only have access to it if the setting allow.user.expunge.rec
over.vm (at account level) is set to true.
(localcloud) > destroy virtualmachine id=<VM_id> expunge=<False or True>
There’s also the expungeV irtualMachine API, that can be used to perma-
nently destroy VMs in the Destroyed, Expunging or Error state.
(localcloud) > expunge virtualmachine id=<VM_id>
Both procedures may be done via UI, as shown next:
Figure 84: Choosing the VM to destroy
85
Figure 85: Confirming VM destruction
:
Figure 86: Choosing the VM to be destroyed
Figure 87: Confirming VM destruction
86
2.3.3. Starting and stopping VMs
Figure 88: Stopping and starting a VM
2.3.4. Restarting a VM
To restart a VM, it must be running.
Figure 89: Choosing the VM to restart
Figure 90: Restarting the VM
2.3.5. Accessing Vms via web interface
Currently, the ACS allows to access the VMs console through the web browser.
For that, there’s a UI button that provide this access in a new tab. However, for
security sake, there’s a behaviour to restrict console access to one at a time.
This way, if the console is being used, another console from the same VM won’t
87
be opened, displaying the error message: Failed to connect to server / access t
oken has expired. For sharing or later access of an VM console, there’s a button
next to it that copies the console URL directly to the clipboard.
Figure 91: Buttons for a VM console access and URL copying
Via CloudMonkey the VM console access it’s possible through the cre ateC
onsoleEndpoint API, in which the parameter to be passed is the VM’s ID, and
returns the console access URL to the web browser.
To obtain the access URL via CloudMonkey, execute the following command:
$ create consoleendpoint virtualmachineid=<VM_id>
Figure 92: Return from the createConsoleEndpoint API
2.3.6. Increasing computational resources from a VM
Despite that this functionality is also supported by the VMware and XenServer
virtualizers, this section will only use KVM when demonstrating how to prepare
the environment to enable changes to computational resources (RAM and CPU)
with the VM running. In the context of the KVM virtualizer, this functionality ex-
ists with some caveats:
88
• The VM must be configured as dynamically scalable;
• The compute offering type of the VM must be of either custom constrained
or custom unconstrained type;
• The VM’s operating system must support dynamic scale for resources;
• It’s not possible to change neither the vGPU or CPU families of the VM
with the same execution, as operating systems commonly don’t support
this kind of substitution while the VM is running.
• It’s not possible to reduce the VM’s resources, only increase.
It’s necessary that the global setting e nable.dynamic.scale.vm, which is set
to false by default, is set to true.
To configure the desired VMs it’s necessary that they are stopped. The nec-
essary definitions for dynamic scalling on KVM are only informed in the Docu-
ment Object Model (DOM) in the VM deploy. Thus, it’s not possible to changes
those definitions while the VM is running.
Already existing VMs that are stopped can be configured as dynamically scal-
able:
Figure 93: Enabling dynamic scaling in a VM
89
It’s also possible to setup templates so that VMs created with them are dy-
namically scalable since their creation.
Figure 94: Configuring dynamic scaling in a template
Lastly, to increase the computational resources of a VM, just use the follow-
ing API command through the CloudMonkey:
scale virtualmachine id=<VM_id> serviceofferingid=<compute_offering_id>
details[0].memory=<new_memory_amount> details[0].cpuNumber=<new_CPU_quantity>
details[0].cpuSpeed=<new_CPU_speed>
For example:
scale virtualmachine id=13661a05-6640-48d3-938f-2ba054356d0b serviceofferingid=daa3225c-caa8-4a37-a534-cf53635aaaf9
details[0].memory=512 details[0].cpuNumber=2
details[0].cpuSpeed=1000
Some importants notes about running the command:
• In the API, the memory values (details[0].memory) and the number of
CPUs (detail s[0 ].cpuNumber) are mandatory. If there’s no intention to
change the amount of memory, just inform the current value, and the
same applies for number of CPUs;
• The memory value (details[0].memory) informed in the API must be the
current value + a multiple of 128.
90
• If the VM has a compute offering of custom unconstrained type, the value
for CPU speed (details[0].cpuSpeed) it’s also mandatory;
Depending on the VM’s operating system, it may be necessary to activate the
CPUs or RAM for them to become usable. On Ubuntu, it’s possible to activate
the CPU in the following way:
1. Become the root user:
sudo su
Activate the CPU with the following command, changing the CPU number
to the desired:
echo 1 > /sys/devices/system/cpu/cpu<CPU_NUMBER>/online
2.3. Repeat those steps for the other CPUs that need to be activated..
On CentOS and RHEL the following procedure can be used to activate RAM:
1. Use this command to check for deactivated RAMs:
grep line /sys/devices/system/memory/*/state
If there’s any deactivated RAM, activate ir with the command;
echo online >/sys/devices/system/memory/memory[number]/state
It’s also possible to increase the computational resources of a VM through
the CloudStack UI, as explained in section 2.18.2.4 (Changing the compute of-
fering of a VM).
2.3.7. Assigning a VM to another account
When using root admin users or domin admin users it’s possible to assign a
VM from one account to another. root admin users can assign VMs to accounts
in any domain, while domain admin users can only assign VMs to accounts
within the same domain and its subdomains
As an example, consider the following domains structure:
91
ROOT
-> domain1
-> subdomain1
-> subdomain2
-> domain2
-> subdomain1
A domain admin from the do main1 can reassign VMs from/to accounts
within the domains ROOT/do main1, ROOT/domain1/subdomain1 and ROO
T/domain1/subdomain2. However, that user won’t be able to reassign VMs
to/from accounts within the domain ROOT/domain2 and its subdomains.
Also, this process have some important caveats:
2.• The VM must be stopped;
• The VM cannot have snapshots;
• The account to which the VM is being migrated must have access to the
networks that the VM uses. Also about that, it’s important to consider
that’s not possible to migrate a network between accounts;
• If the default NIC is connected to a isolated network and the new account
has more than one isolated network, it will be necessary to specify the
network;
• The IPs from the VM cannot have Port Forwarding/Load Balancing rules or
ne Static Nat.
Figure 95: Choosing the VM that will have its account migrated
92
Figure 96: Migrating the VM’s account
It’s also possible to perfom this process using CloudMonkey, described in
section 3, through the following command:
(localcloud) > assign virtualmachine virtualmachineid=<vm-id> domainid=<domain-id>
account=<account-name>
2.3.8. Adding user-data in a VM
The user-data is a script that executes on boot in the VM, and can be added
either during the VM creation or afterwards, updating its definition. Its assign-
ment may be done in two ways, either through CloudMonkey or through the
UI.
Via CloudMonkey, the script may be assigned encoded in base64 format
(because of the CloudMonkey architecture, a base64 maximum size is 2KB),
with the userdata= parameter, followed by the script encoded.
Example os assigning a user-data via CloudMonkey:
(localcloud) > deploy/update virtualmachine [...] userdata=IyEvYmluL2Jhc2gKCmVjaG8gInVzZXJkYXR
hIHRlc3QgLSAkKGRhdGUpIiA+IC90bXAvdXNlcmRhdGE=
The script above encoded:
#!/bin/bash
echo "userdata test - $(date)" > /tmp/userdata
93
Notice that [... ] was used only to improve the visualization of the user-data
parameter, removing all other mandatory parameters.
The updateVirtualMachine API used in example above, in addition to change
the boot settings, can also be used to change other settings, such as memory
values, number of CPUs and CPU speed. So, to prevent that other settings are
changed by the final users, it’s necessary to add some values in the global set-
ting user.vm.denied.details.
These values are the settings names, comma separated. For example, to
block an user to change the settings for memory and CPU, the setting must
have the following value: cpuNumber, cp uSpe ed, memory. It’s important to
highlight that the default value for this setting is: rootdisksize, cpuOvercommi
tRatio, memoryOvercommitRatio, Message.ReservedCapacityFreed.Flag.
Through the UI, the user-data may be added in the text field available, in
which it’s not necessary to encode because the CloudStack itself encodes it
when sending the script to the back-end, and the limit applied it’s of 1048576
characters (1MB size).
Figure 97: Adding user-data when creating the VM
94
Figure 98: Changing user-data when editing the VM
With the user-data added, it’s possible to verify its functionality as show in
the following screenshot:
Figure 99: Verifying the user-data functionality
2.3.9. VM settings
To edit a VM settings, it’s necessary to first stop it. Then, the VM settings
may be changed accessing the Settings tab.
95
Figure 100: VM settings tab
2.3.9.1. General settings
Setting Description.
keyboard Defines the keyboard layout in the VM.
cpu.corespersocket Defines the number of CPU cores per socket.
nicAdapter Defines the virtual network adapter used by the VM.
2.3.9.2. Settings for VMs with custom compute offering
Setting Description
cpuNumber Number of CPU cores assigned to the VM.
cpuSpeed Clock speed of the processor assigned to the VM.
memory Amount of RAM memory assigned to the VM.
96
2.3.9.3. Miscellaneous settings for internal usage
Setting Description
deployvm Action for implanting a new VM.
SSH.PublicKey Public SSH key used to authenticate the VM access.
password Password used to authenticate the VM access.
Encrypted.Password Encrypted password used to authenticate the VM ac-
cess.
configDriveLocation Location where the settings unit from the VM will be
mounted.
2.3.9.4. Settings for importing VMs with NIC, disk and custom parameters for
custom compute offerings
Setting Description
nic Virtual network adapter used by the VM.
network Virtual network used by the VM.
ip4Address IPv4 address assigned to the VM.
ip6Address IPv6 address assigned to the VM.
disk Virtual disk used by the VM.
diskOffering Virtual disk offering used by the VM.
configurationId Id from the configuration used to implant an existing VM in
a new host machine.
2.3.9.5. Adding VM settings via API
Not all VM settings are available through the UI. However, it’s possible to change
or add them through the updateVirtualMachine API, as shown below:
update virtualmachine id=<VM_id> details[0].<property1>=<value1>
details[0].<property2>=<value2> <other parameters>
When executing the updateVirtualMachine command to update the VM set-
tings, only the settings passed through the command will be assigned to the
VM. The current settings will be removed if they aren’t included in it.
(admin) > update virtualmachine id=565f5741-315e-47ea-816b-3f4d98b49553
details[0].keyboard=us details[0].cpu.corespersocket=1
details[0].Message.ReservedCapacityFreed.Flag=true details[0].video.hardware=vga
details[0].io.policy=threads details[0].iothreads=enabled
97
It’s possible that settings passed are mispelled or do not exist, in which case
such settings won’t affect the VM because they will be ignored. However, they
will remain visible in the VM settings tab.
2.4. Managing volumes
There are two types of volumes used by the VMs:
• Root Disk: The disk that has the "kernel" or "base" of the operating system
from a virtual machine is known as "root disk". In Unix operating systems,
this partition is usually designated as "/". Each virtual machine contains
an unique root disk, with no possibility for a VM to have two or more of
this type of disk.
• Data Disk: Data disk unit that expands the available storage capacity for a
virtual machine. A VM may have several data disks attached to it; however,
the maximum quantity vary for each hypervisor.
The difference between these two types is their location within the VM. A
volume that is labeled as Root Disk can become a Data disk and vice versa.
Aside from the volumes different types, they can be in the following states:
• Ready: Volumes in this states already exists in the primary storage and
may be either detached and ready to be assigned to a VM or already in
use by a VM, with their current use percentage available for checking.
• Allocate d: Represents a volume that don’t physically exists yet, however
the volume metadata are already stored in the CloudStack.
• Uploaded: After uploading the volume, it’s stored in the secondary stor-
age. It will only be copied to the primary storage when assigned to a VM.
In this state, the volume may only be attached to a VM as a DATADISK
6
.
6
For more information, check the section 2.4.1.6
98
• Mi g ra t i n g: Transactional state that indicates that the volume is being mi-
grated between storages.
• Destro y: A volume in this state is still available for recovery, or can also
be permanently deleted; in this state the volume cannot be attached to a
VM.
• Expunging: Transactional state that indicates that the volume already went
through the Destroy state, but it’s still being permanently deleted.
Disks generated from a hypervisor, such as KVM, are not compatible with
other hypervisors due to their creation and how they are recognized based on
the specific disk format for each of the hypervisors.
2.4.1. Operations in data disks and root disks
In this subsection are presented some of the operations available for root
disks and data disks. The section 2.18.1 has more details about operations with
root disks.
2.4.1.1. Viewing disks and volumes
Figure 101: Visualizing all the volumes
99
2.4.1.2. Creating a volume
Figure 102: Creating a new volume
Figure 103: Configuring the new volume
Figure 104: Verifying the new volume
All volumes created will be with the Data Disks type.
100
2.4.1.3. Uploading a local volume
Figure 105: Uploading a local volume
Figure 106: Performing the upload
101
To upload a volume, the disk offering 2.18.1 must be of the custom disk type,
since that, during the upload, the disk size is unknown.
2.4.1.4. Uploading a local volume using cURL
By using CloudMonkey and cURL via terminal, it’s also possible to upload local
volumes. For that, follow these steps:
1. In CloudMonkey, use the following command
7
to obtain the parameters
used for next step:
get uploadparamsforvolume format=<volume_format> name=<name> zoneid=<zone_UUID>
Command output example:
{
"getuploadparams": {
"expires": "2022-02-07T15:43:12.591Z",
"id": "41af6492-8695-4b08-b04b-170edfd4f092",
"metadata": "XsUP46nagJgEdN0ZVLHEBMFxJysHre6RtCtGh7+/4/nrK3LvslGsct44QcjIH0sheCig
N9NTn/MElyuv8uxx3V8cA1pc/3KFFZAv9Ycn3XWXBKdUQsKfYe3h/NLnzPv4klpPU35o4h8qjTi+BTTPT
hotakSWWVZybe0gcYKXZGVOHeZOLqL3PMsGC+TumrSFcvW79smfytdMAebNG2nhekVmf8Lwbqdhn0G9Cl
0Szv4NAl4ZIJValMBz+qL84Z/GDEw/V1Nq1EYVqQpoyftlA9ZkhPiitLGeQgJ7UzyHaoMG3vCu0EQgBDe
IgyoJdVRpkw23BR35WTvK2HSIijjOkm/fQJJbMzwBAdLRN1wW+4bPhzLQfc+Jf/ruVk1BDEA3RRDQLp5o
VDfqveV/fj6fpnkjUmyJJ3trDlitb5i/ZnqPGt+2VeJu/UJzNUtDorV/9KVUVf7SEHTL9B154ds1RNAiK
CLM3mfo",
"postURL": "https://172.16.200.99/upload/41af6492-8695-4b08-b04b-170edfd4f092",
"signature": "A8ElmOHx3gAC5xA43b0Kwa97qac="
}
}
2. Via terminal, use the curl command to upload the volume:
curl -X POST "<postURL>" -H "X-signature:<signature>" -H "X-metadata:<metadata>" \
-H "X-expires:<expires>" -H "Expect:" \
-F "file=@<path_to_the_file_for_upload>" -v --insecure
It’s important to notice that the E xpect header must be empty, otherwise
cURL will wait for a signal from SSVM to perform the upload, which in turn will
cause a timeout, as the SSVM don’t implement this function.
The operation for generating a template using a volume is described in the
page 162.
7
Only mandatory parameters are shown. There are other parameters that allow finer con-
trol on how the volume will be created. For example, the parameter account allows to designate
a volume to an account (in this case, the domainid parameter must also be informed).
102
2.4.1.5. Online volume upload via URL
Figure 107: Uploading a volume
Figure 108: Configuring the volume upload
103
2.4.1.6. Adding a volume to a VM
Figure 109: Accessing the volume details
Figure 110: Adding the volume to a VM
104
Figure 111: Configuring the VM that will receive the new volume
2.4.1.7. Removing a volume from a VM
Figure 112: Removing the volume from a VM
Figure 113: Confirming the operation
2.4.1.8. Removing and adding a ROOT volume
Through the CLI it’s possible to both add or remove ROOT volumes, provided
that the VM is stopped. To remove a volume, use the following command:
105
detach volume id=<volume_id>
For adding a volume as ROOT, use the following command, defining its dev
iceid to 0:
attach volume id=<volume_id> virtualmachineid=<VM_id> deviceid=0
The X defines the volume position within the devices list. The ROOT volume
is always in the position 0. When adding a volume that’s not ROOT, the param-
eter may be ommited or changed to another number. The maximum amount
of volumes supported depends on the hypervisor used:
Hypervisor Amount of volumes
KVM 32
VMWare 13
XenServer 6
2.4.1.9. Resizing a data disk volume
This option is available for volumes created by an user, except for volumes that
were created via upload.
Figure 114: Resizing a volume
106
Figure 115: Configuring the resizing
2.4.1.10. Volume download
It’s important to notice that downloading ROOT volumes it’s only possible if the
VM is stopped.
Figure 116: Downloading a volume
Figure 117: Confirming the download
107
2.4.1.12. Removing a volume
Figure 120: Removing a volume
Figure 121: Confirming the operation
2.5. Guest networks
Guest networks are responsible for the internal communication of the VMs,
either between them or with their respective gateways. A guest network op-
tionally can be used as a VPC, in addition to support public IP operations. The
possible operations are:
109
2.5.1. Operations with guest networks
2.5.1.1. Add a guest network
Figure 122: Browsing to the guest network
2.5.1.2. Network types
There are three different types of network that can be created:
• Isolated:
An isolated network that allows only the communication between the VMs
that use it, without the ability to communicate with VMs that run in other
isolated network. The operations related to the use of public IPs are also
applied to this type of network.
110
Figure 123: Rede isolada
Figure 124: Created isolated network
• L2:
Allows that network services are provided and managed by equipments
external to the CloudStack. As a result, the virtual router isn’t needed and
111
VMs implanted in this type of network will get their IP addresses and other
services through these external equipments.
Figure 125: Creating a L2 network
Figure 126: Created L2 network
• Shared:
For the ACS version 4.16, networks of this type can only be created by
root administrators of the cloud. Furthermore, networks of this type can’t
have their IP range changed, with only the option to change its name and
displayname.
112
A shared network executes a single static DHCP service, providing an range
of public IPv4 IP addresses to assign to the instances. Virtual machines
created with this offering will receive a public IPv4, making them directly
available to the internet. A network’s scope may be defined during its cre-
ation as:
– Account: Visible only all users within an determined account;
– Domain: Network available for all accounts within a domain and their
users. Using this setting, it’s also possible to make it visible for sub-
domains as well, making it accessible for all users within the subdo-
mains of the domain that the network was defined;
– Project: Despite visible in the UI, currently it’s not possible to create
shared networks for projects;
– All: All cloud users have access to a network created in this scope.
113
114
Figure 127: Rede shared
Figure 128: Created shared network
115
2.5.1.3. Acesso aos detalhes de uma rede guest
Figure 129: Accessing the details for a guest network
2.5.1.4. Removing networks
Browse to the network details for the network to be removed.
Figure 130: Removing a network
Figure 131: Confirming the network removal
116
It’s only possible to remove a network if there are no VMs using it. Otherwise,
it will be necessary to remove those VMs or change their networks.
2.5.1.5. Editing a network
Figure 132: Editing a network
2.5.1.6. Restarting a network
This options exists only for isolated and shared networks.
Figure 133: Restarting a network
2.5.1.7. Egress rules
For a guest network to be able to communicate externally, it’s necessary that
two egress rules are configured.
117
Figure 134: Configuring the egress rules
• Source CIDR: Address range from the local network to receive access per-
mission.
• Destination CIDR: Address range external to the local network that will be
accessible.
• Protocol: TCP, UDP, ICMP or all.
• Start port e End port: First and last port to be accessed.
2.5.2. Network permissions
When creating a isolated network or L2 network in ACS, the network will
be only visible for the account that owns that network. However, it’s possible
to allow that other accounts within the same domain to have access to the
network through network permissions.
To demonstrate this were created a domain called client, two accounts, clie
nt and client-users, and a network called client-network.
• When accessing the
Guest networks tab witht the client-users account, no
network will be visible.
To allow the account to access the network, will be necessary to create a net
network permission in th client-network network and add the account client-u
sers.
118
• Access the Guest networks menu, select the network and click in Add net
work permissions
Figure 135: Accessing theclient-network network.
• Then, add the account in the network permission
119
Figure 136: Adding an account with Add network permission.
• After adding the account, the network wil be visible.
120
Figure 137: After adding the network permission.
The network permissions can also allow sharing networks between accounts
from the same project. Just create a network permission and add the project.
2.6. Public IPs
The operations that are possible while using public IPs can be accessed get-
ting to the desired public IP details:
121
Figure 138: Accessing the public IP
Figure 139: Viewing the public IP details
2.6.1. Operations with public IPs
2.6.1.1. Firewall setup
A public IP within a guest networks can have its firewall configured to receive
communications.
122
Figure 140: Setting up the firewall
• Source CIDR: Address range external to the network to receive access per-
mission.
• Protocol: TCP, UDP, ICMP or all<u4> Start port e End port: First and last
port to be open to the source CIDR address range.
2.6.1.2. Enabling static NAT
When enabling Static NAt, it’s possible to reserve a public IP for a VM.
Figure 141: Enabling static NAT
123
Figure 142: Configuring the IP
Figure 143: Verifying the final settings
With that, the VM becomes visible and accessible to the external network.
124
2.6.1.3. Enabling port forwarding
When enabling the port mapping it’s possible to allocate specific ports for the
public IP to "redirect" the traffic to another port from a given VM.
Figure 144: Accessing and setting up port forwarding in the ports tab
Figure 145: Choosing the VM that will have a port mapped
125
Figure 146: Example for port forwarding setup with several VMs
The greatest benefit of using port forwarding is turning multiple VMs acces-
sible to the external network through a single IP, in addition to allow to control
which ports from the VMs may be accessed from the external network.
2.6.1.4. Enabling load balancing
Enabling this option makes it possible to perform load balancing operations
between many VMs through a single public IP, following the rules defined by
the user.
Figure 147: Accessing and setting up the ports for load balancing
126
Figure 148: Adding the VMs that will be managed by the load balancer
Figure 149: Created load balancer
Once the load balancer is created, it’s possible to add more complex rules
to it.
It’s important to emphasize that all operations her shown can be easily un-
done by just removing the settings added to the public IP or even removing the
public IP itself.
127
Figure 150: Removing the public IP
2.6.2. Public IP acquisition
To perform operations with public IPs in a network, it’s necessary that a
public IP is allocated to that network. By default, networks managed by ACS
have at least one IP called Source NAT, that can’t be removed.
Guest networks with conserve mode set to false in their network offering
(section 2.18.3) and VPC tiers don’t support more than one service with the
same public IP. They can only have one of the functions amoung Source Nat,
Static Nat, Port Forwarding and Load Balancing.
Therefore, the procedure to acquire a new public IP in a network is:
1. Browse to the desired network:
Figure 151: Accessing the network details
2. Browse to the public IPs tab:
128
Figure 152: Access the network public IP
3. Acquire a new public IP to the network:
Figure 153: Acquire a new IP
Figure 154: List of available IPs
129
Figure 155: New IP acquired
It’s also important to highlight that the available public IPs depends on
three factors:
(a) Public IP range allocated during the Zone creation in ACS;
(b) Quantity of public IPs still available
8
.
(c) And, if the account that is trying to allocate new public IPs has a range
of dedicated IPs, in addition to the global setting use . s y st e m . p ub l i c . i
ps is set to true.
2.7. Virtual Private Cloud (VPC)
A VPC is a network composed from multiple isolated subnetworks, layers
9
,
all connected and managed from the same virtual router. When using a VPC, it’s
possible to establish a virtual network topology that emulates an actual physical
network.
In addition, a single VPC may be used to manage several isolated networks,
making it possible to use it to increase the cloud security.
Here are the operations available when using VPCs and its components:
8
They were not allocated to other networks.
9
Also called tiers.
130
2.7.1. VPC operations
2.7.1.1. Adding a VPC
Figure 156: Criando uma nova VPC
Figure 157: Configuring a new VPC
131
Figure 158: Verifying if the VPC were correctly created
Here it’s important to highlight some details:
• CIDR: defines the range for possible IPs in this virtual network. Tiers allo-
cated in this network must be within this range. Also, a VPC is not acces-
sible out of the zone it was created.
• VPC Offering: The available options are:
– Default VPC Offering:
Services supported: VPN, DNS, Static Nat, Network ACL, User Data, Source
Nat, Lb,Port Forwarding, DHCP. Creates a virtual router but don’t make
it redundant.
– Default VPC Offering with Netscaler:
Performs the same as the Default VPC Offering, however, beucase it
uses Netscaler, adds the possibility to perfom load balancing.
– Redundant VPC Offering:
Performs the same as the Default VPC Offering, however it creates a
redundant virtual router.
By default, when creating a VPC, the Apache CloudStack automatically cre-
ates a virtual router.
132
2.7.1.2. Restarting a VPC
It’s possible to restart a VPC, either for applying more complex changes or to
debug some issue. When perfoming this operation, it’s important to keep in
mind that the VMs that utilizes such VPC will lost connection during the restart
process.
Figure 159: Restarting a VPC
Figure 160: Options for VPC restart
The available options are:
• Clean up: will try to clean old network elements, if any.
• Make redundant: will create a second virtual router in one of the structure
hosts, preferably in a different than the host with the first virtual router.
This option will only show up if the VPC isn’t redundant.
133
2.7.1.3. Removing a VPC
When removing a VPC, it’s necessary to remove the tiers that are part of it. The
virtual routers, however, will be automatically deleted.
Figure 161: Removing a VPC
Figure 162: Confirming the operation
Figure 163: Possible error while trying to remove a VPC
A detailed description on how to remove tiers from a VPC can be found in
the page 118. Once the tiers of the VPC are properly removed, just repeat the
removal process.
2.7.1.4. Public IP acquisition for a VPC
These are the steps to acquire a public IP:
134
Figure 164: Accessing the VPC
Figure 165: Accessing the public IPs of the VPC
Figure 166: Choosing a public IP
135
These steps just allocated a public IP for the VPC, but don’t link them with
any VM. Available operations for public IPs are described in section 2.6.
2.7.2. Tiers
A tier works like an isolated network within the VPC.
2.7.2.1. Adding tiers to a VPC
Figure 167: Accessing the created VPC
Figure 168: Verifying the VPC details
136
Figure 169: Adding a new tier
Figure 170: Creating a new tier
It’s important to highlight that’s not possible to have two tiers with the same
CIDR in the same VPC.
137
With the tier added to the VPC, it’s possible to select them during the cre-
ation of a VM to add it to the tier isolated network.
2.7.2.2. Removing tiers from a VPC:
Figure 171: Browsing to tiers
Figure 172: Removing a tier
Figure 173: Confirming the tier removal
138
If there are any VMs that utilizes this tier, a error will occur and will be neces-
sary to stop and remove
10
all the VMs using that tier
11
and repeat the removal
process.
2.7.3. Network ACL
An ACL (or Network ACL/ACL list)is a set of rules that allow or block traffic
passing through tiers. This rules may be used to grant or restrict communica-
tion between the Internet and the tiers, and also between tiers themselves.
Basically, the ACL functions as a joint from the egress ruler from a guest
network with the firewall of a public IP, but designed for tiers. Each tier has
only one ACL, but an ACL may be used by multiple tiers.
By default, the CloudStack has two options for ACLs that can’t be changed: d
efault_allow, which allows all in and out traffic, and default_deny, which blocks
all traffic. To add a custom rule set, an ACL must be created:
Figure 174: Adding a new ACL
10
Or change the network used by the VM.
11
This process is described in the page 65.
139
Figure 175: Creating a new ACL
Figure 176: Browsing to the created ACL
Figure 177: Browsing to new ACL rule creation
To add an ACL rule, the following fields must be filled:
• #Rule:
Rule number. Rules are evaluated in ascending order, from smallest to the
greatest. If left empty, it will be automatically defined (auto-increased)
140
• CIDR list:
Defines the range of IPs that this rule will be applied.
• Action:
Defines if the rule is a permission or a restriction.
• Protocol:
Available options are: TCP, UDP, ICMP, All e Protocol Number. This gives
great flexibility when creating rules.
• Traffic Type:
Defines the type of traffic, either in or out.
• Description:
Rule description.
Example for creating a rule to setup the SSH command to function properly:
Figure 178: Adding rule to allow traffic coming in through the port 22 from the TCP protocol
(SSH)
141
Figure 179: Adding rule to allow traffic going out through the port 22 from the TCP protocol
(SSH)
Thus, when creating a new tier in that VPC, it’s possible to use the newly
created ACL.
In the time this document wsa written, when creating a ACL through the
CloudStack UI, it could be only used by tiers that belongs to the VPC in which
it was created. However, through the createNetworkACLList API, when leaving
the parameter vpcId blank, it’s also possible to root admin accounts to create
global ACLs that, similarly to the default ACLs (default_allow and default_deny),
are available for all VPCs to use. Example:
create networkacllist name=<acl_name>
2.7.4. Domain VPC
Domain VPC is a feature that allows a VPC to be managed by an account
while the tiers for that same VPC can be created for different accounts, making
it possible to share resources.
142
When using Domain VPCs, several account can use the same VPC and con-
sume only a single VR; each tier will keep isolated via broadcast domain.
To utilize this functionjality, just select the account to own the tier being
created:
Figure 180: Selecting an account to own the tier
To make it possible to create the tier, the account creating the tier and the
account that owns the VPC must have access to the account in which the tier will
be created. The account creating the tier must also have access to the account
that owns the VPC.
2.7.4.1. Allocating public IPs in domain VPCs
If necessary to allocate an public IP to be used by a domain VPC, the public IP
must be acquired to the account that owns the tier that will use the public IP,
and not to the account that owns the domain VPC. If acquired by the account
143
that owns the domain VPC, only that account will be able to use it; accounts that
owns tiers in the VPC will not. The account is selected in the public IP acquisition
form.
Figure 181: Selecting the account in the public IP acquisition form
2.8. Virtual Private Network (VPN)
2.8.1. VPN types
The CloudStack allows creating VPNs for users to access their VMs, with two
possible VPN types:
• C2S (Client-to-Site) VPNs: Allows the user to create safe connections with
the VPC, making it possible to they to connect in their VMs without as-
signing public IPs for them. The CloudStack uses the IPSec protocol for its
connections. It only connects two public IPs at a time, therefore, in this
type of VPN only one user at a time will be able to connect to the VPC
when it’s using the same public IP.
• S2S (Site-to-Site) VPNs: Allows the user to create safe connections between
a datacenter and the VPC, for example, connect an office network with the
cloud.
144
2.8.2. Operations with VPNs
2.8.2.1. Creating a VPN
• To create a VPN, firstly it’s necessary to have a VPC (section 2.7) with a
offering (section ??) that supports the VPN service.. If it already exists, just
add it:
Figure 182: Selecting the VPC
• In the VPC details there’ll be several tabs, however the tab related to the
User VPN is Public IP Addresses. The others tabs, Private Gateway, VPN
Gateway and VPN Connectionare related to S2S VPNs.
Figure 183: VPC details
145
• Information about the VPNs will be located in the Public IP marked as
source-NAT.
• To configure the VPN, browse to the VPN tab from the Public IP:
Figure 184: Public IP details
• In this tab the Enable Remote Access VPN button can be used to enable the
VPN for that VPC:
Figure 185: Habilitando a VPN
To enable the VPN, it’s necessary that the following UDP ports are open:
1. 500 (IKE)
2. 1701 (L2TP)
146
3. 4500 (NAT-T)
• Then, the CloudStack will enable the VPN for the VPC and generate a pre-
shared key to be used in connections with IPSec, as shown in the following
example:
– VPN gateway: 192.168.100.52
– IPSec pre-shared key: kuNbGHMBvODUxztzUXtyDz7q
Figure 186: VPN enabled
• The IPSec pre-shared key size can be changed through the global settings:
Name Description Default
value
remote.access.vpn.psk.length Size of the shared IPSec key
(min: 8, max: 256)
24
2.8.2.2. Disabling a VPN for a VPC
After enabled, it’s possible to disable a VPN for a VPC:
147
Figure 187: Desabling a VPN
2.8.2.3. Managing users
To manage the usres, browse to the VPN users:
Figure 188: Managing VPN users
2.8.2.4. Adding a user
To add an user to a VPN, click in the Add VPN user button and fill the form
fields:
Figure 189: Starting to add a VPN user
148
Figure 190: Adding a VPN user
2.8.2.5. Defining maximum VPN users per account
It’s possible to set the maximum quantity of VPN user for each account
may have through the global settings:
Name Description Default
value
remote.access.vpn.user.limit Maximum number of VPN
users per account
8
2.8.2.6. Removing an user
After created, an user may not be changed, only removed:
Figure 191: Removing a VPN user
149
After configuring the VPN and adding a user, the connection may be es-
tablished.
2.8.2.7. Connecting to the VPN via Linux
In the network settings, a new VPN must be added:
Figure 192: Starting to add a VPN on Linux
The protocol used bu CloudStack is the L2TP.
In the form, the VPN’s gateway, the suer and the registered password must
be informed.
Figure 193: Adding a VPN on Linux
The IPSec pre-shared key must be informed:
150
Figure 194: Informing the IPSec pre-shared key
Figure 195: Informing the IPSec pre-shared key
Also, the EAP protocol must be disabled, because CloudStack don’t support
it:
Figure 196: Disabling the EAP protocol
151
Figure 197: Disabling the EAP protocol
After adding the VPN, the connection can be established.
2.8.2.8. Connecting to the VPN via Windows
Under the network and internet settings, browse to the VPN tab and add a new
VPN connection:
Figure 198: Adding a VPN on Windows
152
The following form will open:
Figure 199: Starting to add a VPN on Windows
Important settings:
• Name or server address: enter the VPN gateway.
• VPN type: select the option L2TP/IPSec with pre-shared key.
• Pre-shared key: inform the IPSec pre-shared key.
Figure 200: Finishing to add a VPN on Windows
153
In the fieldInput information type, select the option Username and pass-
word and then fill the form’s fields with the user and password previously reg-
istered.
After adding the VPN, it’s necessary to change some settings:
1. Go to Network connections and open the VPN adapter properties.
2. Options → Properties → PPP settings → and check all options.
Figure 201: Changing the PPP settings
3. Security → Allow these protocols → Microsoft CHAP Version 2.
154
Figure 202: Enabling the MS-CHAP v2 protocol
4. Networking → Disable IPv6.
Figure 203: Disabling IPv6
5. Right button IPv4 → Properties → Advanced → Ip Settings →
Disable "Use default gateway on remote network".
155
Figure 204: Disabling the use of default gateway in the remote network
Finally, establish connection with the VPN.
2.8.3. S2S (Site-to-Site) VPNs
To setup a S2s VPN connection, first it’s necessary to create a VPN customer
gateway:
Figure 205: Browsing to the VPN customer gateway tab
The following creation form will open:
156
Figure 206: Creating a VPN customer gateway
Just inform the gateway, the CIDR (if more than one, separating with com-
mas) and the client’s IPSec pre-shared key It’s also possible to change some
security settings on the remaining fields if necessary.
After creating a VPN customer gateway, it’s possible to edit it:
157
Figure 207: Editing a VPN customer gateway
And, if the VPN customer gateway isn’t part of any connection, it’s also pos-
sible to remove it:
Figure 208: Deleting a VPN customer gateway
The next step is create a VPN gateway to the VPC that will be part of the VPN
S2S connection. In the VPC details, browse to the VPN gateway tab and search
for the Create site-to-site VPN gateway button:
Figure 209: Browsing to the VPN gateway under the VPC details
When clicking on the button, the CloudStack will provide a IP address that
158
will be de VPC gateway from the VPC.
Figure 210: Creating a VPN gateway
Lastly, it must be created the VPN S2S connection on the VPC. Still on the
VPC details, go to the VPN connection tab and search for the Create site-to-site
VPN connection button:
Figure 211: Browsing to the VPN connection under the VPC details
The following form will open:
159
Figure 212: Creating a VPN S2S connection
Inform the VPN customer gateway and, if connecting two VPCs, check the
option Passive in the VPC that won’t start the connection.
After created it’s possible to restart the connection:
Figure 213: Restarting a VPN S2S connection
It’s also possible to remove it:
Figure 214: Removing a VPN S2S connection
160
2.9. Signing HTTP requests
The Cloudstack API requires a signing process to send HTTP requests, with
the goal to properly authenticate and authorize the operator. For this, the ACS
can provide api keys and secret keys, that will be used in the request body
during the procedure.
• Generating keys via UI:
161
Figure 215: Generating keys.
• Generating keys via CloudMonkey:
register userkeys id=<user_id> filter=<desired_filters>
• Checking keys via CloudMonkey
162
get userkeys id=<user_id> filter=<desired_filters>
The request sent to the CloudStack API can be either GET or POST nad have
the following body:
• CloudStack main URL, for example:
http://example.com:8080/client/api
• The command that will be executed, for example: listUsers
• Parameters required by the command
• The API key that will be used to identify the user
• The signature used to authenticate the user
• Request example
http://localhost:8080/client/api? command=<command>&
<parameter(s)>&
apiKey=<api_key>&
signature=<signature>
All request will be of the same format URL + Command + Signature; to gen-
erate the signature, follow these steps:
1. Separate each of the request’s fields with a & and apply URLencode
2. Make sure that the command strings are completely in lowercase, for ex-
ample:
apikey=mivr6x7u6bn_sdahobpjnejpgest35exq-jb8cg20yi3yaxxcgpyuairmfi_ejtvwz0nukkjbpmy3y2bcikwfq&
command=deployvirtualmachine&
diskofferingid=1&serviceofferingid=1&
templateid=2&zoneid=4
3. Then, it’s necessary to calculate the command string hash using the SHA-1
hashing algorithm with the user’s secret key;
4. Lastly, enconde the resulting key in Base64, and the result will be some-
thing similar to the following example:
163
http://localhost:8080/client/api?
command=deployVirtualMachine&serviceOfferingId=1&
diskOfferingId=1&
templateId=2&
zoneId=4& apiKey=miVr6X7u6bN_sdahOBpjNejPgEsT35eXq-jB8CG20YI3yaxXcgpyuaIRmFI_EJTVwZ0nUkkJbPmY3y2bciKwFQ&
signature=Lxx1DM40AjcXU%2FcaiK8RAP0O1
To illustrate the process of signing requests in practice, a step-by-step guide
of how to perfom this procedure is shown below:
1. Import the modules needed:
>>> import urllib2
>>> import urllib
>>> import hashlib
>>> import hmac
>>> import base64
2. Specify the ACS endpoint, commands to be executed and the user keys:
>>> baseurl='http://localhost:8080/client/api?'
>>> request={}
>>> request['command']='listUsers'
>>> request['response']='json'
>>> request['apikey']= 'plgWJfZK4gyS3mOMTVmjUVg-X-jlWlnfaUJ9GAbBbf9EdM-kAYMmAiLqzzq1ElZLYq_u38zCm0bewzGUdP66mg'
>>> secretkey= 'VDaACYb0LV9eNjTetIOElcVQkvJck_J_QljX_FcHRj87ZKiy0z0ty0ZsYBkoXkY9b7eq1EhwJaw7FF3akA3KBQ'
3. Create the request string:
>>> request_str='&'.join(['='.join([k,urllib.quote_plus(request[k])]) for k in request.keys()])
4. Start the signature encryption process with HMAC, Base64 and URL en-
conding:
>>> sig_str=
'&'.join(['='.join([k.lower(),urllib.quote_plus(request[k]).lower().replace('+','%20')])
for k in sorted(request.iterkeys())])
>>> sig_str
'apikey=
plgwjfzk4gys3momtvmjuvg-x-jlwlnfauj9gabbbf9edm-kaymmailqzzq1elzlyq_u38zcm0bewzgudp66mg&
command=listusers&
response=json'
>>> sig=hmac.new(secretkey,sig_str,hashlib.sha1)
>>> sig
<hmac.HMAC instance at 0x10d91d680>
>>> sig=hmac.new(secretkey,sig_str,hashlib.sha1).digest()
>>> sig
'M:]x0exafxfbx8fxf2yxf1px91x1ex89x8axa1x05xc4Axdb'
>>> sig=base64.encodestring(hmac.new(secretkey,sig_str,hashlib.sha1).digest())
>>> sig
'TTpdDq/7j/J58XCRHomKoQXEQds=n'
>>> sig=base64.encodestring(hmac.new(secretkey,sig_str,hashlib.sha1).digest()).strip()
>>> sig
'TTpdDq/7j/J58XCRHomKoQXEQds='
>>> sig=urllib.quote_plus(base64.encodestring(hmac.new(secretkey,sig_str,hashlib.sha1).digest())
.strip())
164
5. Lastly, assemble the string and send the request:
>>> req=baseurl+request_str+'&signature='+sig
>>> req
>>> 'http://localhost:8080/client/api?
apikey=plgWJfZK4gyS3mOMTVmjUVg-X-jlWlnfaUJ9GAbBbf9EdM-kAYMmAiLqzzq1ElZLYq_u38zCm0bewzGUdP66mg&
command=listUsers&
response=json&
signature=TTpdDq%2F7j%2FJ58XCRHomKoQXEQds%3D'
>>> res=urllib2.urlopen(req)
>>> res.read()
'{ "listusersresponse" :
{ "count":3 ,"user" :
[
{"id":"7ed6d5da-93b2-4545-a502-23d20b48ef2a",
"username":"admin",
"firstname":"admin",
"lastname":"cloud",
"created":"2012-07-05T12:18:27-0700",
"state":"enabled",
"account":"admin",
"accounttype":1,
"domainid":"8a111e58-e155-4482-93ce-84efff3c7c77",
"domain":"ROOT",
"apikey":
"plgWJfZK4gyS3mOMTVmjUVg-X-jlWlnfaUJ9GAbBbf9EdM-kAYMmAiLqzzq1ElZLYq_u38zCm0bewzGUdP66mg",
"secretkey":
"VDaACYb0LV9eNjTetIOElcVQkvJck_J_QljX_FcHRj87ZKiy0z0ty0ZsYBkoXkY9b7eq1EhwJaw7FF3akA3KBQ",
"accountid":"7548ac03-af1d-4c1c-9064-2f3e2c0eda0d"},
{"id":"1fea6418-5576-4989-a21e-4790787bbee3",
"username":"runseb",
"firstname":"foobar",
"lastname":"goa",
"email":"joe@smith.com",
"created":"2013-04-10T16:52:06-0700",
"state":"enabled",
"account":"admin",
"accounttype":1,
"domainid":"8a111e58-e155-4482-93ce-84efff3c7c77",
"domain":"ROOT",
"apikey":
"Xhsb3MewjJQaXXMszRcLvQI9_NPy_UcbDj1QXikkVbDC9MDSPwWdtZ1bUY1H7JBEYTtDDLY3yuchCeW778GkBA",
"secretkey":
"gIsgmi8C5YwxMHjX5o51pSe0kqs6JnKriw0jJBLceY5bgnfzKjL4aM6ctJX-i1ddQIHJLbLJDK9MRzsKk6xZ_w",
"accountid":"7548ac03-af1d-4c1c-9064-2f3e2c0eda0d"},
{"id":"52f65396-183c-4473-883f-a37e7bb93967",
"username":"toto",
"firstname":"john",
"lastname":"smith",
"email":"john@smith.com",
"created":"2013-04-23T04:27:22-0700",
"state":"enabled",
"account":"admin",
"accounttype":1,
"domainid":"8a111e58-e155-4482-93ce-84efff3c7c77",
"domain":"ROOT",
"apikey":
"THaA6fFWS_OmvU8od201omxFC8yKNL_Hc5ZCS77LFCJsRzSx48JyZucbUul6XYbEg-ZyXMl_wuEpECzK-wKnow",
"secretkey":
"O5ywpqJorAsEBKR_5jEvrtGHfWL1Y_j1E4Z_iCr8OKCYcsPIOdVcfzjJQ8YqK0a5EzSpoRrjOFiLsG0hQrYnDA",
"accountid":"7548ac03-af1d-4c1c-9064-2f3e2c0eda0d"} ] } }'
165
It’s important to mention that it’s also possible to define a expiration time
for API calls. The server will monitor the time frame specified and then reject all
requests sent after the expiration period. To enable this feature it’s necessary
to add the following parameters to the request:
Parameter Value Description
signatureVersion 3 If this parameter is not
equals to 3 or is empty, it
will be irgnored in the re-
quest.
expires YYYY-MM-DDThh:mm:ssZ(ISO 8601) Specifies the datetime for
the signature on the re-
quest to expire.
Usage example:
• expires=2011-10-10T12:00:00+0530
• Example for a request with expiration time:
http://<IPAddress>:8080/client/api?
command=listZones&
signatureVersion=3& expires=2011-10-10T12:00:00+0530&
apiKey= miVr6X7u6bN_sdahOBpjNejPgEsT35eXq-jB8CG20YI3yaxXcgpyuaIRmFI_EJTVwZ0nUkkJbPmY3y2bciKwFQ&
signature= Lxx1DM40AjcXU%2FcaiK8RAP0O1hU%3D
166
2.10. Templates and ISOs
Templates are reusable images that serves as a model or standard for creat-
ing VMs. Intrinsically, a template is a image from a virtual disk that contains its
basic configurations, such as operating system, users, default passwords, and
event advanced configurations, such as softwares previously installed and se-
tupped. It’s worth to highlight that templates depends on the hypervisor used.
During the template creation, the user must define if it will be private or
public. When set to public, the template will be available to all users that can
access the zone where the template was stored. the new template will become
visible in the Templates menu when the template creation process is finished,
i.e., when the ACS finishes its download and preparation. The template then
become available for creating a new VM.
ISOs are images from optical disks, such as CD, DVD or Blu-ray. It’s a file with
an exact copy, or image, from the whole optical disk content. These images are
frequently used to provide huge programs or operating systems, allowing that
the user burn the image to a physical disk or mount it as a virtual disk in their
system to access its content without the need of a physical disk.
Just like templates, ISOs may be set as public or private. An important detail
about ISOs is that they don’t depends on specific hypervisors, so, the same
image may be mounted in different hypervisors, unlike templates. VMs created
with this resource must be configured since their installation.
Similarly to templates, ISOs can be stored and provided with certain privacy
settings. During its registration, the user will label the image as bootable or non
bootable. The bootables are the images that contain a operating system.
2.10.1. Operations with templates/ISOs
The Apache CloudStack offers many operation options for templates and
ISOs, such as:
167
2.10.1.1. Viewing
Figure 216: Viewing templates and ISOs
• Template:
Figure 217: Accessing the template details
168
Figure 218: Template details
• ISO:
Figure 219: Accessing the ISO details
169
Figure 220: ISO details
2.10.1.2. Listing
By default, the graphical interface sends the all filter (which lists all registered
elements in the environment) when the request is sent by either a root admin
or domain admin account. For accounts of the user type the request is sent with
the self filter (which lists only ISOs and templates that were registered by the
user). These filters may be changed clicking in the text field next to the refresh
button.
The following filters are available to use:
• featured: Returns elements marked as featured;
• self : Returns elements registered by the user that sent the request;
170
• selfexecutable: Similar to the previous filter but includes only images that
may be used to instance new VMs;
• sharedexecutable: Returns the elements that were created by other users
but that the user sending the request have permission granted;
• executable: Returns the elements that the user sending the request reg-
istered as well as elements labeled as public that can be used to instance
new VMs;
• community: Returns the elements labeled as public that were not marked
as featured;
• all: Returns all elements from the environment (can only be used by ad-
ministrators)
Figure 221: Listing filters for ISOs and templates.
171
2.10.1.3. Registry
• Template:
Figure 222: Registering the template
172
Figure 223: Configuring the template
– Direct download: if selected the template will be directly download
to the primary storage when the VM is first created. Then, it will be
stored in the primary storage as cache and will be removed only when
all VMs that utilizes it were destroyed. This way, a new download will
be only made in case that all the VMs that uses the template were
removed.
Templates marked with this option will have the Bypassed Secondar
173
y Storage state, indicating that they were directly downloaded to the
primary storage and will never be stored in the secondary storage.
• ISO:
Figure 224: Registering the ISO
Figure 225: Configuring the ISO
It’s necessary to indicate the type of operating system from the template/ISO,
as this information will be used by the hypervisor. If the chosen operating sys-
tem don’t match the template/ISO, some inconsistencies may appear during
the VMs creation.
174
2.10.1.4. Upload
• Template:
Figure 226: Uploading the template
Figure 227: Configuring the template
175
• ISO:
Figure 228: Uploading the ISO
Figure 229: Configuring the ISO
2.10.1.5. Editing
Browse to the template/ISO details.
176
• Template:
Figure 230: Editing the template details
Figure 231: Editing the template
• ISO:
177
Figure 232: Editing the ISO details
Figure 233: Editing the ISO
2.10.1.6. Editing template/ISO permissions
Changes to templates/ISOs permissions may be done via either UI or API. Via
API, the command updateTem platePermissions or updateIsoPermissions can
be used, with the capabilities to remove, add or redefine all of the template/ISO
permissions.
• Template:
178
Figure 234: Editing the template permissions
• ISO:
Figure 235: Editing the ISO permissions
Figure 236: Editing permissions
The options for editing permissions are the same for both templates and
ISOs. Operations available:
• Add:
Used to add the template/ISO to a given account or project.
• Remove:
Used to remove the template/ISO from a given account or project.
179
• Reset:
Reset all the template/ISO permissions to default settings.
2.10.1.7. Editing the template sharing
Browse to the template/ISO details.
• Template:
Figure 237: Editing the template sharing
• ISO:
Figure 238: Editing the ISO sharing
Figure 239: Editing the sharing options
180
The global setting allow.user.view.all.domain.accounts, which is set to false
by default, makes so that regular users don’t see the list of all existing accounts
within the domain that they are part of, forcing them to know exactly the name
of the account to share the template/ISO. There will be a field for the requesting
user to provide the recipient account name.
The options for editing details about the sharing is the same for both tem-
plates and ISOs.
2.10.1.8. Removing a template/ISO
Templates/ISOs can be removed but there are some caveats. The default be-
haviour for CloudStack is to refuse removing a template/ISO when there are
VMs using it.
After removing templates/ISOs, VMs instances in the same zone that uses
that image won’t be affected and will keep executing. However, the removed
template/ISO won’t be available for new instances.
• Template:
Figure 240: Deleting a template
• ISO:
181
Figure 241: Deleting an ISO
2.10.1.9. Creating a template based in a existing VM
To create a template based in a existing VM, it needs to be in the Stopped state:
Figure 242: Browsing to the VMs
182
Figure 243: Selecting a volume from the VM
Figure 244: Creating a template
183
Figure 245: Configuring a template
2.11. Resource Tags
Tags, or resource tags, are pairs in the key:value format that store metadata
about some resource to categorize it.
It’s very useful for filtering lists and for custom CloudStack Usage reports
(such as Quota activation rules, section 4).
To add or remove tags:
• Via UI:
184
Figure 246: Resource tags from a given component
• Via CloudMonkey: For this, the listTags, crea teTags and deleteTags APIs
can be used. In the deleteTags API, the parameter tags[0] can be ommited
for removing all tags from the resource.
A host, storage pool or offering may also have resource tags (which shouldn’t
be confused with host tags or storage tags).
2.12. Affinity Groups
Affinity groups is a functionality that allows the user define which VMs must
run, or not, in the same host. These groups may be used, for example, to ensure
a low latency connection between VMs when grouping them in the same host or
specify VMs that provide the same service to run in different hosts to improve
the system’s fault tolerance, allowing that if a host fails, the VMs that are in
other hosts will keep running and providing the service normally.
Affinity groups shouldn’t be mistaken for Instance groups, which in turn are
only used for the user to organize their VMs, grouping them, and don’t affect
in which hosts the VMs will run.
2.12.1. Affinity groups types
There’re four affinity groups types going forward from CloudStack version
4.18:
12
:
12
In versions previous to 4.18, only the host affinity and host anti-affinity types existed, which
are equivalent to host affinity (strict) and host anti-affinity (strict) types, respectively.
185
• Host affinity (strict): specify that the VMs must always run in the same
host.. However, if the host don’t have enough capacity to run all the VMs
from the group, an error during the creation of new instances will occur.
• Host anti-affinity (strict): specify that the VMs must always run in differ-
ent hosts. Similarly, if there are not enough hosts for all VMs, an error will
occur when creating new instances.
• Host affinity (non-strict): specify that the VMs should run in the same
host whenever is possible. If the host don’t have enough capacity, new
instances will be created in some other host.
• Host anti-affinity (non-strict): specify that the VMs should run in different
hosts whenever is possible. If the there isn’t enough hosts, new instances
will be created in a host that already have others instances.
2.12.2. Affinity groups operations
2.12.2.1. Adding an affinity group
Figure 247: Browsing to the affinity groups
186
Figure 248: Creating an affinity group
Figure 249: Affinity group created
2.12.2.2. Adding a VM to an affinity group
The VM must be stopped for the option to move it to the affinity group appear.
Figure 250: Accessing the VM that will be moved to the group
187
Figure 251: Moving a VM to an affinity group
Figure 252: Selecting the group
2.12.2.3. Viewing VMs in the affinity group
Figure 253: Accessing the group
188
Figure 254: Viewing the VMs in the group
Figure 255: List of VMs in the group
189
2.12.2.4. Removing an affinity groups
Figure 256: Removing the group
2.13. Autoscale VM groups
The autoscale VM groups functionality allows the creation of groups for VMs
that automatically increase or decrease in number as the VMs are demanding
more or less resources. This functionality can be used to, for example, add
more machines that offer a service when all the existing ones are under a huge
load or to remove them when demand lowers, both cases without the need of
intervation from an operator. Tha way, it makes possible a more efficient usage
of computational resources and, consequently, reducing costs.
This automatic management for instances works through the creation of
two set of rules: one for determine when new VMs must be created and other to
determine when they need to be removed. For example, it’s possible to define
that new machines will be added when the average CPU usage of all instances
are greater than 70% and that they will be removed when lower than 40%.
To perform this management, CloudStack monitors usage statistics for the
190
instances provided pelo hypervisor
13
. Furthermore, it uses a load balancer to
distribute the usage between the VMs. Therefore, the groups must associated
to a isolated network or to a VPC with the load balancing service enabled and
configured.
2.13.1. Creating am autoscale VM group
Before creating an autoscale VM group it’s necessary to create an isolated
network or a VPC and setup the load balancing service. Below the setup of an
isolated network is exemplified:
1. Create an isolated network:
Figure 257: Creating an isolated network
13
The VMware, XenServer and KVM hypervisors all support the autoscale VM groups function-
ality.
191
2. Access the Public IP addresses tab and acquire a new public IP for the
network:
Figure 258: Accessing the Public IP addresses tab
Figure 259: Acquiring a new public IP
3. In the public IP, accessing the Load balancing tab and adding a load bal-
ancing
14
:
14
The setting Autoscale rule indicates that the rule will be used by an autoscale VMs group.
192
Figure 260: Adding a load balancing rule
With the isolated network setupped it’s possible to proceed to the autoscale
VM group creation menu:
1. Configure the group VMs through zone, template, compute offering and
data disk settings.
2. Select the isolated network created:
Figure 261: Selecting the isolated network
3. Select the load balancing rule created:
193
Figure 262: Selecting the load balancing rule
4. Setup the scale-up policy:
The scale-up policy is a set of rules that determines when new machines
must be added to the group. It has the following settings:
• Condition: rules that define when new VMs must be added to the
group. When more there’s more thant one condition, all must be
true to the scape-up happen;
• Duration: amount of time, in seconds, that the rules must maintain
the true value to add VMs;
• Quiet Time: after VMs are added, amount of time that the scale-up
policy won’t add more instances.
Each condition has the following parameters:
• Counter: statistic from the group VMs that must be monitored. The
counters available are:
– VM CPU - average percentage: average CPU usage from the VMs,
in percentage (0 a 100);
– VM Memory - average percentage: average main memory usage
from the VMs, in percentage (0 a 100);
– Public Network - mbps received per vm: average amount of data
being received by the VMs, in Mbps;
– Public Network - mbps transmit per vm: average amount of data
being sent by the VMs, in Mbps;
194
– Load Balancer - average connections per vm: average number of
connections per VM.
• Operator: operator used to compare the values;
• Threshold: value to compare with the counter.
Figure 263: Setting up the scale-up policy
In the image, for example, a scale-up policy was created to add new VMs
when de current average CPU usage kept greater than 10% for 30 sec-
onds.
2.13.1.1. Setting up a scale-down policy
The scale-down policy follows the same idea that the scale-up policy, but
determines when that VMs must be removed.
195
Figure 264: Setting up a scale-down policy
In the image, a scale-down policy was defined to remove VMs when the
average CPU usage kept lower than 10% for 60 seconds.
2.13.1.2. Configuring autoscale VM group details
• The settings are the following:
– Name: group name;
– Expunge VM grace period: amount of time, in seconds, waited af-
ter a scale-down before removing the VMs. This wait time helps
to guarantee that all pending sessions and transactions were fin-
ished.
– Max members: max number of VMs in the group;
– Min members: minimum number of VMs in the group;
– Polling interval: interval, in seconds, that the conditions are veri-
fied.
196
Figure 265: Configuring the autoscale VM group details
2.13.1.3. Group creation
After creating a group of autoscale VMs, the instances in it may be visualized
in the following way:
Figure 266: Visualizing the instances from a autoscale VM group
197
Figure 267: Instances from the autoscale VM group
2.13.2. Testing the autoscaling
With the instances created, it’s possible to test the autoscaling functionality.
To force an increase in CPU usage and, consequently, a scale-up, the instances
may be accessed and then executed the following command:
for i in 1 2 3 4; do while : ; do : ; done &done
This command will create four processes with a infinite loop. Each loop gen-
erates a 100% usage in one of the CPU cores.
After a few minutes, the scale-up will be perfomed and the group will have
reached the maximum amount of instances.
Figure 268: Instances in the autoscale VM group after the scale-up
To perform the scale-down, the processes shall be finished:
sudo killall -9 bash
Then, after a few minutesm, the numeber os instances will be reduced to
minimum amount.
198
Figure 269: Instances in the autoscale VM group after the scale-down
2.13.3. Editing an autoscale VM group
To edit an autoscale VM group it’s necessary to deactivate them first:
Figure 270: Deactivating the autoscale VM group
Figure 271: Confirming the operation
199
While the group is deactivate, VMs will keep running normally. However,
neither scale-up or scale-down will be performed based on the demand.
After deactivated it’s possible to edit the group details, VM deploy parame-
ters and scale-up and scale-down policies:
Figure 272: Editing group details and VM deploy parameters
Figure 273: Dialog screen for editing group details and VM deploy parameters
200
Figure 274: Editing the group scale-up policy
Figure 275: Editing the group scale-down policy
After the changes are done, activate the group again:
201
Figure 276: Activating the autoscale VM group
Figure 277: Confirming the operation
2.13.4. Removing an autoscale VM group
It’s also possible to remove an autoscale VM group:
Figure 278: Removing the autoscale VM group
202
Figure 279: Confirming the operation
2.14. Kubernetes
The Kubernetes Service plugin adds Kubernetes integration to the Cloud-
Stack. Such plugin is disabled by default and only admin users can enable
ir through the global setting cloud.kubernetes .service.enabled. It allows that
users execute services in containers using Kubernetes clusters.
Since version 4.16 from ACS, the Kubernetes Service plugin uses the default
system VM template for the Kubernetes cluster implantation. Moreover, it’s
used an ISO for installing the Kubernetes binaries in the cluster nodes, in a way
that, for each Kubernetes version desired to use, it’s necessary to provide a
specific ISO to the CloudStack.
For implanting and configuring the Kubernetes cluster nodes, the plugin uti-
lizes the kubeadm tool from Kubernetes. The kubeadm it’s a command line
tool to easily provide a Kubernetes cluster in physical servers, cloud or virtual
machines. Internally, the control nodes starts a Kubernetes cluster using the
command kubeadm init with a custom token and the worker nodes join this Ku-
bernetes cluster using the command kubeadm join with the same token. More
information about the kubeadm can be found in the official documentation.
The Weave Net CNI it’s used for virtual networks within the cluster. More infor-
mation about the Weave Net CNI can be found in the official documentation.
The plugin allows creating Kubernetes clusters using either UI or API. Both
UI and API provide functionalities to list, remove, update, stop and start these
clusters, as well as increase or decrease the numbers of nodes in them.
203
2.14.1. Kubernetes versions supported
The ACS allows that the user keep multiple Kubernetes versions. To upload
a Kubernetes version it’s necessary to create an ISO containing these files:
1. Kubernetes binary files
2. CNI binary files
3. CRICTL binary files
4. Network settings files from Weave
5. Dashboard settings files from Kubernetes
2.14.2. Adding a Kubernetes ISO to the Apache CloudStack
The community provides some ISOs ready to use. Tests and examples in
this document were done with the ISO containing the version 1.23.3. However,
given the fast release cycle of the Kubernetes ISOs, it’s recommended to use the
latest version. It’s important to notice that operations over Kubernetes ISOs are
only available to root admin users.
Down below there’s a compatibility matrix between the ISOs and the ACS:
K8s
ACS
4.18.0
< 1.23.3
1.24.0
1.25.0
1.26.*
1.27.*
1.28.4
Since version 4.16, Kubernetes started to use the SystemVM (Debian) tem-
plate for cluster deployment, and the SSH user changed to cloud.
When creating a cluster, although that the ACS allows names containing up-
percase letters, Kubernetes only works with lowercase names. Therefore, when
204
creating Kubernetes clusters it’s necessary to name the cluster using only low-
ercase characters. Otherwise, when trying to perfom operations that demands
communication with CloudStack, Kubernetes won’t find the cluster nodes.
Through the addKubernetesSup portedVersion API it’s possible to add an
Kubernetes ISO to ACS.
add kubernetessupportedversion name=example mincpunumber=2 minmemory=2048 semanticversion=1.23.3 \
url=<ISO_download_URL> zoneid=<zone_id>
It’s also possible to perfom the same operation via UI, accessing the Kuber-
netes ISOs page, in the images submenu; clicking in the button "Add Kuber-
netes version", the form below will show up:
Figure 280: Form for adding a Kubernetes version
205
2.14.3. Removing a Kubernetes ISO
Through the deleteKubernetesSupportedVersion API it’s possible to remove
a Kubernetes ISO from ACS.
delete kubernetessupportedversion id=<ISO_id> filter=<optional>
The same operation can also be done via UI, accessing the Kubernetes ISOs
page, in the Images submenu; clicking in the ISO do delete; and then clicking in
the delete button.
Figure 281: Deleting Kubernetes ISOs via UI
Figure 282: Confirming the removal of a Kubernetes ISO
2.14.4. Kubernetes clusters
A Kubernetes cluster consists of two main components: control node and
work nodes. The control node is responsible to organize and manage the clus-
206
ter functioning, with the possibility to be replicated to provide more safety to
the system. Work nodes are the nodes that the applications run. The nodes
may be VMs or physical machines, which contain pods, the smallest unit within
a Kubernetes cluster. They work as creation templates for a container type.
The Kubernetes Service plugin provides functionalities to create and man-
age Kubernetes clusters, making container implantations easier and without
the need to manually setup the Kubernetes in every container node. The ser-
vice will automatically provide the desired number of virtual machines accord-
ingly with the cluster size, using binaries corresponding to the given Kubernetes
version. Beyond that, the service offers functionalities to update the cluster and
increase/decrease the number of nodes in it. These, when executing, can be
updated to a newer minor or patch version of Kubernetes, as well as increased
or decreased.
2.14.4.1. Creating Kubernetes clusters
The plugin provides functionalities to create Kubernetes clusters for shared or
isolated networks in the CloudStack.
The createKubernetesCluste r API allows to create Kubernetes clusters and
its usage it’s illustrated just below.
create KubernetesCluster name=example description=example zoneid=<zone_id> \
kubernetesversionid=<Kubernetes_ISO_id> serviceofferingid=<compute_offering_id> \
size=<number_of_work_nodes>
The same operation in available via UI, accessing the Kubernetes page, in
the Compute submenu; clicking in the Create Kubernetes cluster button. the
following form will show up:
207
Figure 283: Form for creating a Kubernetes cluster
There are some details that must be given attention when creating Kuber-
netes clusters:
• Currently there must be only one Kubernetes cluster per network.
• The Kubernetes ISOs, when created, have a minimum resource require-
ments, such as CPU cores and allocated memory. Because of this, it’s nec-
essary that a compute offering that fulfill all those requirements exists.
• When creating a Kubernetes cluster, at least two VMs will be instantiated
with the selected compute offering : one for the control node and other
as a work node. This number is increased based on the cluster size used
208
during its creation. For that, it’s worth highlight that the resources usage
will be greater than the minimum requirements.
• To access the nodes via SSH it’s necessary to inform a SSH keys pair during
the Kubernetes cluster creation. Furthermore, the user for connecting via
SSH is cloud
15
.
2.14.4.2. Scaling of Kubernetes cluster
The scaleKubernetesCluster API allows scaling a Kubernetes cluster, increasing
or decreasing the number of work nodes in it, or changing the compute offering
user for the cluster. However, the compute offering can only be increased. An
example for the API usage is shown below.
scale KubernetesCluster id=<Kubernetes_cluster_id> size=<new_cluster_size> \
serviceofferingid=<compute_offering_id>
The same API must be used via UI, as shown below.
Figure 284: Form for scaling a Kubernetes cluster
15
Until version 4.16 from ACS, the user for connecting was core.
209
2.14.4.3. Upgrading a Kubernetes cluster
The upgra deKubernetesCluster API allows to upgrade the minor or patch ver-
sion of a Kubernetes cluster in execution. For that, it’s necessary that an ISO
with the desired version it’s already available in ACS.
upgrade kubernetescluster id=<Kubernetes_cluster_id> kubernetesversionid=<Kubernetes_version_id>
The same API must be used via UI, as shown below.
Figure 285: Form for upgrading a Kubernetes cluster
It’s important to notice that the Kubernetes version upgrade can only be
performed between patch versions from the current minor version or a minor
version to the next minor; never jumpimg versions. This restriction also applies
to ACS.
2.14.5. Kubernetes access
While the Kubernetes cluster is created, an user interface is automatically
generated for the cluster. The Kubernetes documentation, available in this link,
shows more information about the Kubernetes dashboard. The UI provides an
Access tab with information about how to access the Kubernetes, including a
step-by-step guide.
210
Figure 286: Accessing the Kubernetes
To give the cluster access to the dashboard, the user needs to first run a local
proxy using kubectl and the kubeconfig file. For security sake, logins based with
kubeconfig are not allowed, being necessary to generate an access token and
using the kubectl to make the connection. The command below can be used to
run the proxy, provided the proper path to the kubeconfig file:
kubectl --kubeconfig /custom/path/kube.config proxy
As soon as the proxy is running, the dashboard is accessible clicking in the
available link. The token used for login can be obtained using the following
command:
kubectl --kubeconfig /custom/path/kube.config describe secret $(kubectl --kubeconfig
211
/custom/path/kube.config get secrets -n kubernetes-dashboard |
grep kubernetes-dashboard-token | awk '{print $1}') -n kubernetes-dashboard
2.14.6. Generating a token for the dashboard
A default user token can’t be used to access the Kubernetes dashboard. To
access it, it’s necessary to create a token for the kubernetes-dashboard user in
the kubernetes-dashboard namespace and inform them when logging in. The
token can be created using the following command:
kubectl --kubeconfig <caminho-arquivo-kube.conf> create token kubernetes-dashboard -n kubernetes-dashboard
2.15. Snapshots
Snapshots are recovery points for a whole VM or just a single disk. They
can be used as a way to recover from failures or even to create templates and
new volumes. In some hypervisors, such as VMware and KVM, they behave like
backups.
2.15.1. Snapshot types
There are two types for snapshots:
2.15.1.1. Volume/Disk snapshot
Take a snapshot from the disks
16
of a VM. The cloud administrator can define
a limit for snapshots that a user may have. Users may create volumes and
templates based in a snapshot.
It’s possible to create snapshots manually or automatically, through set-
tings. They are created in the Primary Storage and, if the global setting sna
pshot.bac kup.to.sec ondary is set to true
17
, when done, are transferred to the
Secondary Storage, where they will be kept until their removal.
16
The section 2.4 brings information about VM disks in more detail.
17
We recommend to always use this setting set to true
212
2.15.1.2. VM snapshot
Takes a snapshot similarly to the volume snapshot, however this type of snap-
shot has options to include memory/CPU states of the VM. This procedure can
make faster VM restores easier
18
. This type of snapshot supports incremental
snapshots. From the first snapshot of a VM, the following snapshots will save
their differences in a delta file
19
.
Some limitations that snapshots have:
• Changes to volumes, CPU or memory to a VM that has snapshots may
cause errors, either trying to perform these changes or trying to restore
the VM to the snapshot.
• It’s not possible to save a copy of two volumes of a VM in the same snap-
shot.
• It’s not possible to perform a snapshot from a VM and a volume at the
same time.
• Only snapshots created through the CloudStack are managed by it.
2.15.2. Snapshot operations
The operations that are available for snapshots are:
18
As long as no changes to memory/CPU in this VM are made.
19
This function depends on the virtualizer used, so, for more information check the virtual-
izer documentation.
213
2.15.2.1. Volume snapshot
Figure 287: Accessing the VM details
Figure 288: Taking a snapshot of the volume
Figure 289: Choosing which volume to take a snapshot
214
Figure 290: Configuring the snapshot
2.15.2.2. Vm snapshot
Figure 291: Taking a snapshot of the VM
Figure 292: Configuring the snapshot
215
2.15.2.3. Viewing created snapshots
Figure 293: Accessing the snapshots
Figure 294: Accessing the volume snapshots
Figure 295: Accessing the VM snapshots
216
2.15.2.4. Creating a template from a volume snapshot
Figure 296: Accessing the snapshot details
Figure 297: Creating a template
217
Figure 298: Configuring the template information
2.15.2.5. Creating a volume from a volume snapshot
Figure 299: Creating a new volume
218
Figure 300: Configuring the new volume
2.15.2.6. Reverting a volume snapshot
Figure 301: Reverting to the snapshot
Figure 302: Confirming the operation
219
2.15.2.7. Removing a volume snapshot
Figure 303: Removing a volume snapshot
Figure 304: Confirming the removal
2.15.2.8. Creating a volume snapshot from a VM snapshot
Figure 305: Accessing the VM snapshot details
220
Figure 306: Creating a volume snapshot from a VM snapshot
Figure 307: Configuring which volume will be extracted
Figure 308: Creating a volume snapshot
221
2.15.2.9. Reverting a VM snapshot
Figure 309: Reverting the snapshot
Figure 310: Confirming the operation
2.15.2.10. Removing a VM snapshot
Figure 311: Removing a VM snapshot
222
Figure 312: Confirming the operation
2.15.2.11. Configuring recurrent snapshots
This operation is described in the ?? page.
2.16. Events
Events are generated when a user performs an action in the cloud or when
the state of a resource, either virtual or physical, is changed.
Using events, is possible to monitor task, jobs and processes in CloudStack,
including their possible errors .
2.16.1. Event types
The possible event types are:
• INFO: informs when a monitored process executed correctly.
• WARN: informs when a monitored process found some kind of problem,
yet it wasn’t terminal
• ERROR: informs that a terminal error occurred in a monitored process.
2.16.2. Searching and filtering events
To visualize and search through the events in the web interface it’s necessary
to:
223
Figure 313: Browsing through the events
The web interface don’t support filtering events by date.
2.16.3. Removing or archiving events
When selectiong the checkbox of an event, it’s also possible to remove or
archive them:
Figure 314: Archiving or removing an event
Regardless of the action selected, a confirmation screen will pop-up.
224
2.17. User interface (UI)
The goal of this topic is to bring in more detail information about the Apache
CloudStack UI, in addition to provide tips to make its usage easier.
2.17.1. Using the browser’s timezone
All dates handled by ACS are stored as in standard UTC in the database and
when querying these data via API, they will also return in UTC. To make the
user experience more pleasant, the GUI has the Use local timezone option, that
when enabled makes so that datetimes informed in the APIs and displayed to
the user are converted to the same timezone as the browser.
Figure 315: Enabling the setting
An example for the usage of this option is when filtering VM statistics. With
the option X disabled, an user inside Brasília’s timezone, for example, would
have to insert values for time intervals from a different timezone than their
own to obtain the desired statistics. For example, if that user is trying to check
for a VM statistics and inserted the time interval from 12:00h to 17:00h with-
out enabling the option, they will receive statistics corresponding to the period
from 9:00h às 14:00h, cause the time would be interpreted in UTC. With the
option enabled, the UI automatically makes the conversion to the browser’s
timezone, allowing that the user query and visualize statistics without making
manual timezone conversions.
This option also affect tariff manipulation and Quota reports.
225
2.17.2. UI documentation
Most of the screen/pages from the CloudStack web interface has small help
icons, which when hovering gives a small description about that component,
that can be quite useful.
There are also question mark (?) icons that can be clicked to open a new
browser tab in the section from the CloudStack official documentation about
that component.
Figure 316: Example for pop-ups from help icons
2.18. Service offerings
To make it possible to create VMs, virtualizers require specification for their
characteristics, such as CPU, memory size or disk to allocate, among others def-
initions. In CloudStack these VM characteristics are grouped and standardized
in form of service offerings, in order to ease VM specification, resource usage
monitoring and, if applicable, charge their usage. CloudStack separates service
offerings in two segments:
• Disk offerings
• Compute offerings
2.18.1. Disk offerings
Disk offerings are service offerings destined to storage resources from a VM.
Virtual machines created based in a template will use disk offerings for creating
extra disks, while those created with ISOs will need a disk offering for creating
the root disk.
226
2.18.1.1. Creating a disk offering
To create a disk offering:
Figure 317: Starting to create a new disk offering
When clicking in the Add Disk Offering button, a form for providing the in-
formation needed will show up:
227
Figure 318: Creating a new disk offering
228
About some of the form fields:
• Provisioning type:
1. Thin Provisioning: disks of this tipe will be allocated with the minimum
necessary size and will be able to grow accordingly with their need,
until reaching a defined limit.
2. Sparse Provisioning: disks of this type will start allocated with the size
established in the offering, however, they may contain previous data.
Their creation is fast, cause the previous data will not be erased or
overwritten, however it’s necessary to do this before writting new
data. The performance for the first writes on this type of disk will
be low.
3. Fat Provisioning: disks of this type will start allocated with the size es-
tablished in the offering and all space used will be overwritten, eras-
ing any data present on the blocks. Their creation will be slower when
compared to sparse provisioning, due to the cleaning of old data,
however, their performance during the first writes will be better.
• Disk Size Strictness: When set to true, the size of the volume cannot be
changed.
• QoS Type: defines the QoS for the disks. To make use of the hypervisor or
storage QoS, it’s necessary that they support this functionality.
1. None: By default, offerings don’t implement it.
2. Hypervisor: Defines the rate of write and read by the hypervisor. De-
pending on the hypervisor, some QoS definitions may not be imple-
mented, therefore, it’s necessary to validate them before applying to
production environment.
3. Storage: Defines a minimum and maximum IOPS for the storage.
229
• Hypervisor Snapshot Reserve: Allows to reserve space to snapshot beyond
the data disk. Measured in percentage of data disk, the final value re-
served is the percentage chosen plus the own data disk size. This doesn’t
apply to KVM.
• Storage Tags: Storage tags will be used to direct the volumes to compatible
storages.
• About the access:
1. Public: Defines if the service offering will be available to all domains
or if its use will be restricted. Select yes for the access in all domains.
2. Domain: This option will only show up when Public is unchecked. This
option determines the domains that will use the offering.
3. Zone: Defines in which zones the offering will be available.
2.18.1.2. Editing a disk offering
After created, only a few attributes of the disk offering may be changed.
• Edit:
Figure 319: Starting to edit the disk offering
230
Figure 320: Editing the disk offering
• Update access:
Figure 321: Starting to update the disk offering access
Figure 322: Updating the disk offering access
• Change order:
231
Figure 323: Changing the disk offering order
2.18.1.3. Removing a disk offering
To remove a disk offering just access it and click on the trash can icon:
Figure 324: Starting to remove the disk offering
Figure 325: Removing the disk offering
If there are any volumes on the disk offering, they will keep working nor-
mally, as well as the migration processes and VM restarting, because the Cloud-
Stack copies the disk offering characteristics to the disk, however it will not be
possible to create new volumes with it.
More information in the official documentation.
232
2.18.2. Compute offerings
Compute offerings are offerings dedicated to computational resources of a
VM, like CPU and RAM memory. A compute offering also possess a disk offering,
that defines the characteristics of the VM’s root disk (when created based in a
template).
When creating a compute offering, it will be associated with a disk offering.
This association can occur in two ways. One of them is through disk specifi-
cations, storage type and tags, which may be included directly in the compute
offering. The second way is to connect a disk offering to a compute offering,
making that the disk offering used by the root disk during the instance creation.
It’s worth to emphasize that users may use a different disk offering for the root
disk during the VM creation.
2.18.2.1. Creating a compute offering
To create a compute offering:
233
Figure 326: Starting to create a new compute offering
When clicking in the Add Compute Offering button, a form for providing the
information needed will show up:
234
Figure 327: Creating a new compute offering - 1 - continues
235
Exaplanation about some of the form fields:
• Compute Offering Type: alternatives that control the autonomy that the
final user may have to create new compute offerings. There are three
options for it:
1. Fixed offerings: offerings with fixed values for CPU (cores and speed)
and memory. All VMs created with this offering will have the same
setting.
2. Custom constrained offerings: in this type of offering the final user
will use the allocated resources within a value range defined by the
administrator. The user that utilizes this offering will have to select
a value within the defined limits for each parameter during the VM
creation.
3. Custom unconstrained offerings: unlimited offerings. Any value may
be informed during the Vm creation, provided that it’s within the value
available for use.
• CPU cores: quantity of cores allocated to attend a VM that uses this service
offering. If the Custom constrained offerings option is selected, it’s neces-
sary to determine the minimum and maximum quantity of cores during
the offering creation. If the Custom unconstrained option is selected, this
field won’t be shown.
• CPU (in MHz): determines the cores speed. This definition it’s only utilized
if the CPU limit is active. If the Cus tom Uncons t r ained option is selected,
this field won’t be shown.
• Memory (in MB): defines the amount of memory, in megabytes, that the
system VM must allocate. If the Custom constrained option is selected,
it’s necessary to determine the minimum and maximum amount of mem-
236
ory during the offering creation. If the Custom Unconst rained option is
selected, this field won’t be shown.
• Host tags: these will be used to direct VMs to compatible hosts.
• Network Rate: defines the data transfer rate, in MB per second.
• Offer HA: If selected, the VM will be monitored and in case of host failure,
it will be restarted in another available host.
• Dynamic Scaling Enabled: If selected, the CPU and memory of the in-
stance may be scaled dynamically.
• CPU Cap: limits CPU usage even if there are available resources.
• Volatile: VMs created with a volatile compute offering will have their root
disks restored to their initial state every time that they are booted.
• Deployment Planner: Algorithm used by CloudStack to choose in which
host to implant the VMs created with this service offering
1. First Fit: selects the host that has enough capacity to support the
instance requirements.
2. User Dispersing: uniformly disperses the instances from a single ac-
count in different clusters or pods.
3. User Concentrated: concentrates instances from a single account in
a single same pod.
4. Implicit Dedication: will implant instances that are dedicateds to a
domain or specific account. If you choose this planner, it will be also
necessary to define a value for the planner mode parameter.
5. Bare Metal: it’s used with a bare metal host.
• Planner Mode: This option will only be available if implicit dedi cation is
selected. The planner mode will determine how the instances deploy will
be made in the private infrastructure.
237
1. Strict: A host won’t be shared between different accounts.
2. Preferred: The instances will be implanted in a dedicated infrastruc-
ture whenever possible. Otherwise, the instance may be implanted
in a shared infrastructure.
• GPU: Assigns a physical GPU, or part of it, to an instance. This makes that
graphical applications can be executed in the VM. A list of available GPUs
is displayed.
2.18.2.2. Editing a compute offering
After the compute offering is created, just a few of its attributes can be changed.
• Edit:
Figure 328: Starting to edit the compute offering
Figure 329: Editing the compute offering
• Updating the access:
238
Figure 330: Starting to update the compute offering access
Figure 331: Updating the compute offering access
• Changing order:
Figure 332: Changing the compute offering order
2.18.2.3. Removing a compute offering
To remove a compute offering, access it and click on the trash can icon:
Figure 333: Starting the compute offering removal
239
Figure 334: Removing the compute offering
If there’s system VMs running under the compute offering they will keep
working normally, as CloudStack copies their compute offering characteristics
to the VM, although won’t be possible to create new ones with it. When live
scaling, the VM that uses that offering will need to exchange it for another one.
2.18.2.4. Changing the compute offering of a VM
It’s also possible to change the compute offering used by an existing VM. There
are two ways to perform this change: via Ui and via CloudMonkey, and for both
ways the VM must be stopped.
• Via UI:
Figure 335: Starting to change the compute offering
240
Figure 336: Changing the compute offering
Figure 337: Changing the compute offering
After performing these steps the compute offering of the VM will be up-
dated.
• Via CloudMonkey: It’s possible to use the APIs scaleVirtualMachine or change-
ServiceForVirtualMachine
20
. In this situation, both share the same be-
haviour and will have the same result. Just use the command:
scale virtualmachine id=<VM_id> serviceofferingid=<compute_offering_id>
20
The syntax used by the APIs is the same, changing only the API call from scale virtualma c
hine to change serviceforvirtualmachine.
241
When successfully executing this command, its result will show several in-
formation related to the VM changed, including the new compute offering
defined.
{
"virtualmachine": {
"account": "admin",
"affinitygroup": [],
"cpunumber": 1,
"cpuspeed": 1000,
"created": "2022-03-08T17:06:23+0000",
"details": {
"Message.ReservedCapacityFreed.Flag": "true",
"rootDiskController": "osdefault"
},
"displayname": "VM-ea2c34dd-a8a9-4b6a-b6c4-9404e034f3b7",
...
"serviceofferingid": "ab647165-7a0a-4984-8452-7bfceb036528",
"serviceofferingname": "Small Instance",
"state": "Stopped",
"tags": [],
"templatedisplaytext": "Macchinina - Light SO",
"templateid": "b81aab72-2c37-466e-b53f-bdaf398322fa",
"templatename": "Macchinina",
"userid": "af1818b9-26de-11ec-8dcf-5254005dcdac",
"username": "admin",
"zoneid": "8b2ceb16-a2f2-40ea-8968-9e08984bdb17",
"zonename": "sc-floripa-01"
}
}
When performing changes to the compute offering of a VM, it’s important
to take into account their host tags and storage tags, in addition to the desired
compute offering tags. This is because only the compatible compute offerings
will be shown as options for VM scaling, i.e., only compute offerings with the
same tags as the VM or those without any tag restrictions.
More information in the official documentation.
2.18.3. Network offerings
Network offerings are offerings destined to network resources, such as DNS,
DHCP, firewall, load balancing, among others services.
2.18.3.1. Creating a network offering
To create a network offering:
242
Figure 338: Starting to create a network offering
By clicking on the Add Network Offering button, a form for inserting the
information needed will open:
Figure 339: Creating a new network offering - 1 - continues
It’s possible to create networks of three different types:
243
• Isolated: Networks designated for a single account.
• L2: Networks without services. When using this type of network, the final
user is responsible for IP management, or static IPs must be used. This
kind of network has a limitation at logical level, making it possible that
it’s created for just one account, however, when necessary that different
accounts use it, it’s possible to create a network of shared type and with
no services, simulating an L2 network(when performing the creation of an
network with this offering, the data for the IPv4 fields are mandatory by
CloudStack, however they will not be used and therefore anything can be
informed)
• Shared: Networks that may be shared between accounts within the same
domain.
The Specify VLAN options allows that the cloud administrator assigns a spe-
cific VLAN to the created network. It’s important that this specific VLAN is not
part of the range available for dynamic allocation, as described in the physical
network for guest traffic (it can be consulted at the Physical Network tab from
the corresponding zone). If not selected, the CloudStack will assign a VLAN
chosen at random from the available ones when the network change to the Im-
plemented state and will remove it when the network is no longer being used
(Allocated state). For L2 networks, the VLAN is assigned automatically.
Networks with network offerings marked as Persistent will be implemented
without the need t add a VM to the network.
244
Figure 340: Creating a new network offering - 2
• With the Internet Protocol it’s possible to specify which IP version the of-
fering will support. The CloudStack offers two options: IPv4 (default) and
Dual Stack. The Dual Stacj allows that the offering supports both IPv4 and
IPv6.
• The options Promiscuous Mode, MAC Address Changes and Forged Transmits
are destined to use with Nested Virtualization, on VMWare.
• The Supported Services are the services available for selection. Each ser-
vice has a configuration, making ir possible to choose a provider for each
of them.
• From the compute offering, it’s possible to inform the service offering that
the VR will utilize. For using this functionality, it’s necessary that at least
one of the following services is enabled in the list of Supported Services:
VPN, DHCP, DNS, Firewall, LB, UserData, SourceNat, StaticNat, PortFor-
warding.
245
• Networks with Conserve Mode will allocate resources only when the first
VM is created.
• It’s possible to select the Enable network offering option for the created
network offering become ready for use. Otherwise, it will be necessary to
enabled it before using it.
2.18.3.2. Editing a network offering
After created, only a few attributes of the network offering may be changed.
• Edit:
Figure 341: Starting to edit the network offering
Figure 342: Editing the network offering
• Enable/Disable the network offering:
246
Figure 343: Starting to disable the network offering
Figure 344: Disabling the network offering
If there is any network using the offering when disabling it, those networks
will keep working normally, although will not be possible to create new
networks.
• Updating access:
Figure 345: Starting to update the network offering access
Figure 346: Updating the network offering access
247
• Changing order:
Figure 347: Changing the network offering order
2.18.3.3. Removing a network offering
To remove a network offering just access it and click on the trash can icon:
Figure 348: Starting the network offering removal
Figure 349: Removing the network offering
If there are networks using the network offering or it’s a default offering,
won’t be possible to remove it, although it can still be disabled.
More information in the official documentation.
248
2.18.4. VPC offering
VPC offerings are offerings destined for VPC (more information about VPCs
in section 2.7). They are similar to network offerings (section 2.18.3)
2.18.4.1. Creating a VPC offering
To create a VPC offering:
Figure 350: Starting to create a new VPC offering
By clicking on the Add VPC Offering button, a form for inserting the infor-
mation needed will open:
249
Figure 351: Creating a new VPC offering
The Supported Services are the services available for selection. Each service
has a configuration, making ir possible to choose a provider for each of them.
For the most part, only the VpcVirtualRouter provider exists.
The Compute offering field is explained in Section 2.18.3.
then, it’s possible to select the Enable VPC offering option for the created
VPC offering become ready for use. Otherwise, it will be necessary to enabled
it before using it.
2.18.4.2. Editing a VPC offering
After created, only a few attributes of the VPC offering may be changed.
• Edit:
250
Figure 352: Starting to edit a VPC offering
Figure 353: Editing a VPC offering
• Enable/Disable the offering:
Figure 354: Starting to disable the VPC offering
Figure 355: D disabling the VPC offering
If there is any created VPC while the offering is being disabled, the VPC will
keep working normally, although will not be possible to create new ones.
251
• Updating access:
Figure 356: Starting to update the VPC offering access
Figure 357: Updating the VPC offering access
• Changing order:
Figure 358: Changing the VPC offering order
2.18.4.3. Removing a VPC offering
To remove a network offering just access it and click on the trash can icon:
Figure 359: Starting the VPC offering removal
252
Figure 360: Removing the VPC offering
If there are VPCs using the VPC offering or it’s a default offering, won’t be
possible to remove it, although it can still be disabled.
253
3. CloudMonkey, API and others
The CloudMonkey is a command line tool (Command Line Interface - CLI)
written in Go and is used for interacting with the CloudStack API. It’s a open-
source project, that can be accessed through its github.
The steps for installation, setup and use of the CloudMonkey are shown in
the next subsections.
3.1. Installation
The CloudMonkey have two different ways to be installed, one for Linux/Mac
users and other for Windows users.
• Linux/Mac:
It’s necessary to download the latest release available, choosing the binary
accordingly to the platform used, and copy it to a directory that’s a mem-
ber of the $PATH variable, such as /usr/local/bin. Via terminal:
sudo wget <file link> -O /usr/local/bin/cmk sudo
chmod +x /usr/local/bin/cmk
• Windows:
In the releases link previously mentioned, it’s necessary to download the
executable (.exe) accordingly to the platform used and copy it to a direc-
tory that belongs to the Windows executables directory, such as C:\Wind
ows\System32\. Then, it’s necessary to verify the execution of the cmk.ex
e.
3.2. Setup
By default, CloudMonkey will create an initial profile with some default cre-
dentials from CloudStack. To setup the CloudMonkey, use the command se
t:
$ cmk
Apache CloudStack CloudMonkey 6.1.0
Report issues: https://github.com/apache/cloudstack-cloudmonkey/issues
(localcloud) > set url http://myprovider.cloud/client/api
254
(localcloud) > set username myusername
(localcloud) > set password mypassword
(localcloud) > sync Discovered 622 APIs
For creating a new profile, the same set can be used:
(localcloud) > set profile newprofile
(newprofile) > set url http://myprovider.cloud/client/api
(newprofile) > set username myusername
(newprofile) > set password mypassword
(newprofile) > sync Discovered 622 APIs
It may be necessary to also disable the setting for SSL certificate verification
when using HTTP instead of HTTPS:
(localcloud) > set verifycert false
When logging in with another user, it’s recommended to always use the syn
c command, to find out which APIs that this user have access. To return to the
previous user, just execute the command:
(newprofile) > set profile localcloud
3.3. Usage
The CloudMonkey make calls to the CloudStack API, therefore to use it, it’s
necessary to have some knowledge about the desired operations to perform
and which APIs will be needed. For that CloudMonkey have some facilitators:
• Verifying for available APIs:
(localcloud) > list apis
{
"api": [
{
"description": "Creates VPC offering",
"isasync": true,
"name": "createVPCOffering",
"params": [
{
"description": "desired service capabilities as part of vpc offering",
"length": 255,
"name": "servicecapabilitylist",
"required": false,
"since": "4.4",
"type": "map"
},
...
}
],
"count": 622
}
255
• Autocomplete: using the TAB key a list of command suggestions will show
up.
Figure 361: Utilizando o autocomplete
Figure 362: Checking the autocomplete for a specific command
Figure 363: Checking the autocomplete for a specific API
256
• Details about a API: it’s possible to obtain a detailed description for a spe-
cific API with the -h flag:
Figure 364: Using the -h flag to get help
Or through the help:
(localcloud) > help createAccount
Once that the procedures needed to be done are known, just make the API
call via CloudMonkey:
257
Figure 365: Performing the API call
258
4. Resources consumption accounting
The module for accounting computational resources consumption is subdi-
vided in two mechanisms: usage collection (Usage) and use accounting (Quota),
with the second acting complementarily to the first.
The Usage mechanism periodically performs the identification and collec-
tion of usage data from the environment resources. The Quota mechanism is
a plugin that allows the management of a tariff model over the computational
resources consumption.
This section describes how to access the resource usage reports from Usage,
and active tariffs and reports from Quota
4.1. Usage
4.1.1. Resource usage report
The resource usage reports can be accessed through the list U s a g e R e c o r d s
API, as shown next:
(admin) > list usagerecords startdate='yyyy-MM-dd'T'HH:mm:ssZ' enddate='yyyy-MM-dd'T'HH:mm:ssZ'
This command will return a list containing several entries with the following
structure:
...
{
"account": "<conta>",
"accountid": "<Identificador da conta>",
"description": "<Descrição do relatório>",
"domain": "<nome do dominio>",
"domainid": "<ID do dominio>",
"enddate": "yyyy-MM-ddTHH:59:59+0000",
"rawusage": "depende do usagetype",
"startdate": "yyyy-MM-ddTHH:00:00+0000",
"tags": [<conjunto de tags>],
"usage": "<depende do usagetype>",
"usagetype": <tipo de recurso contabilizado>,
"zoneid": "<ID da zona>"
},
...
When a event is sent to CloudStack, it won’t be immediately visible in the
API requests table, instead the table will be periodically updated, based on the
time period chosen by the operator through the global settings.
260
4.2. Quota
4.2.1. Tariffs
To visualize active tariffs for the user account:
Figure 367: Active tariffs listing
Other information from tariffs, such as name, description and value, can be
accessed in the details page for each tariff.
Figure 368: Tariffs details
4.2.2. Reports
Selecting the Summary submenu and then selecting the desired user ac-
count, it’s possible to visualize details and Quotas reports related to that ac-
count.
261
Figure 369: Accessing details and reports from an account
4.2.2.1. Credits
To visualize the credit addition/removal history (who added, how much was
added and when), browse to the Credits tab:
Figure 370: Credits listing
This listing may also be performed via CloudMonkey. Example:
(admin) > quota creditslist accountid=af16aaed-26de-11ec-8dcf-5254005dcdac domainid=52d83793-26de-11ec
-8dcf-5254005dcdac
262
{
"count": 3,
"credit": [
{
"credit": -192.87,
"creditedon": "2023-10-04T13:05:13+0000",
"creditorid": "af16aaed-26de-11ec-8dcf-5254005dcdac",
"creditorname": "admin",
"currency": "$",
"postingdate": "2023-10-04T13:05:06+0000"
},
{
"credit": 350,
"creditedon": "2023-10-04T13:20:24+0000",
"creditorid": "af16aaed-26de-11ec-8dcf-5254005dcdac",
"creditorname": "admin",
"currency": "$",
"postingdate": "2023-10-04T13:20:24+0000"
},
{
"credit": -25,
"creditedon": "2023-10-05T15:56:58+0000",
"creditorid": "af16aaed-26de-11ec-8dcf-5254005dcdac",
"creditorname": "admin",
"currency": "$",
"postingdate": "2023-10-05T15:56:58+0000"
}
]
}
4.2.2.2. Resource usage
The Quota usage tab show information about resource usage during the period
selected:
263
Figure 371: Resource usage listing
Via CloudMonkey, the consult must be performed as the following example:
(admin) > quota statement account=admin domainid=52d83793-26de-11ec-8dcf-5254005dcdac startdate=2023-1
0-01T00:00:00+0000 enddate=2023-10-05T13:55:01+0000
{
"statement": {
"account": "admin",
"accountid": "af16aaed-26de-11ec-8dcf-5254005dcdac",
"currency": "$",
"domain": "52d83793-26de-11ec-8dcf-5254005dcdac",
"enddate": "2023-10-05T13:55:01+0000",
"quotausage": [
{
"name": "RUNNING_VM",
"quota": 0,
"type": 1,
"unit": "Compute*Month"
},
{
"name": "ALLOCATED_VM",
"quota": 195,
"type": 2,
"unit": "Compute*Month"
},
...outros recursos...
{
"name": "NETWORK",
"quota": 0,
"type": 30,
"unit": "Compute*Month"
264
},
{
"name": "BACKUP_OBJECT",
"quota": 0,
"type": 31,
"unit": "GB*Month"
}
],
"startdate": "2023-10-01T00:00:00+0000",
"totalquota": 202.64381816
}
}
It’s also possible to perform a detailed consumption consult:
Figure 372: Detailed listing for resource usage
Via CloudMonkey:
(admin) > quota statement account=admin domainid=52d83793-26de-11ec-8dcf-5254005
dcdac startdate=2023-10-04T00:00:00+0000 enddate=2023-10-05T13:55:01+0000
showresources=true type=6
{
"statement": {
"account": "admin",
"accountid": "af16aaed-26de-11ec-8dcf-5254005dcdac",
"currency": "$",
265
"domain": "52d83793-26de-11ec-8dcf-5254005dcdac",
"enddate": "2023-10-05T13:55:01+0000",
"quotausage": [
{
"name": "VOLUME",
"quota": 7.64381816,
"resources": [
{
"displayname": "usage",
"quotaconsumed": 4.805108,
"removed": false,
"resourceid": "f710ed0f-c234-4c31-89ae-de23c9d8779f"
},
{
"displayname": "DATA-220",
"quotaconsumed": 2.83871016,
"removed": false,
"resourceid": "e4727094-2571-4ebb-aca7-5cc97778fde0"
}
],
"type": 6,
"unit": "GB*Month"
}
],
"startdate": "2023-10-04T00:00:00+0000",
"totalquota": 7.64381816
}
}
4.2.2.3. Balance
The Dai ly Qu ota b alance tab shows the daily balance data (credits + resource
usage) for the selected period. The balance is closed daily and represents the
sum of all previous data, with the current balance always referring to the last
day balance.
Example: R$100,00 of a resource usage is generated daily. At the end of
the first day, the balance will be -R$100,00. At the end of the second day, the
balance will be -R$200,00. At the end of the third day, the balance will be -
R$300,00. When filtering the search period for the third day to the fifth day,
the balances returned will be -R$300,00 (third day), -R$400,00 (fourth day) e
-R$500,00 (fifth day).
266
Figure 373: Quota balance listing
Via CloudMonkey:
(lab) > quota balance account=oliver domainid=7efa7a0c-8f43-4564-b8cc-d709424dc69a startdate=2022-04-1
0 enddate=2022-04-16
{
"balance": {
"currency": "$",
"dailybalances":
[
{
"balance": 99.67,
"date": "2022-04-10T00:00:00+0000"
},
{
"balance": 99.6,
"date": "2022-04-11T00:00:00+0000"
},
{
"balance": 99.53,
"date": "2022-04-12T00:00:00+0000"
},
{
"balance": 99.47,
"date": "2022-04-13T00:00:00+0000"
},
{
"balance": 94.23,
"date": "2022-04-14T00:00:00+0000"
},
{
"balance": 88.96,
"date": "2022-04-15T23:59:59+0000"
},
267
{
"balance": 83.7,
"date": "2022-04-16T23:59:59+0000"
}
]
}
}
268
5. Cloud-init
cloud-init is a open source software created to automate both starting and
setting up operating systems during boot, focused in cloud virtual machines.
Among its many functionalities are: password management, SSH key injection
and user metadata injection. It’s supported by the biggest cloud orchestration
systems, such as Azure, Google Compute Engine, OpenStack and CloudStack.
5.1. Initialization steps
cloud-init have five initialization steps, wich are:
• Generator: available only for systems that use systemd, this is the first step
to execute and defines if cloud-init should be executed during boot. To
disable the cloud-init execution during the operating system boot needs
only to exist the /etc/cloud/cloud-init.disabled file.
• Local: step that locate the local datasources and apply the network set-
tings on the system.
• Network: this step is executed after the Local step and requires that the
network settings are working. All modules informed in the section cloud-
init-modules from the file /etc/cloud/cloud.cfg will be processed in it. This
step perform the virtual machine initial setup, such as defining the host-
name and executing the mount command.
• Config: this step is executed after the Network step and during it all mod-
ules informed in the section cloud-config-modules from the file /etc/cloud/
cloud.cfg will be processed. It’s designated to independent settings mod-
ules, that don’t affect other modules, such as changing user password,
location and time zone of the operating system.
• Final: last step to be executed during the operating system boot. All mod-
ules informed in the section cloud-final-modules from the file /etc/cloud/c
269
loud.cfg will be processed in it. It’s designated for processes that are usu-
ally executed by the user, such packages installation or update and scripts
execution.
If the user wants to execute their own scripts, they need to put them on
the same /var/lib/cloud/scripts directory, which have three subdirectories
to represent the scripts execution frequency.
– per-boot: scripts in this directory will be executed every time the op-
erating system is booted.
– per-instance: scripts under this directory will be executed only dur-
ing the first boot of the operating system. More information in the
section 5.2.
– per-once: scripts inside this directory will be executed only once exe-
cuted only once, evenif the operating system is changed. More infor-
mation in the section 5.2.
5.2. Execution frequency
The cloud-init have three execution frequencies for modules and scripts:
• per-boot/always: executes every boot.
• per-instance: executes only during the operating system first boot. To de-
termine if it’s the first boot of the operating system, cloud-init stores a
cache for future validations. If the cache exists, means that cloud-init was
executed at least once, therefore this isn’t the first boot. This behaviour
may cause inconsistencies when a image is created based in that instance,
therefore cloud-init adopts a default behaviour, called check, that causes
it to validate the instance ID stored in the cache. If they differ, it wil con-
sider as the first boot. There’s also a trust behaviour that skip the instance
ID validation. The instance ID usually is defined by the orchestrator when
270
a new virtual machine is created. It’s also possible to clear the cache man-
ually to refined the first boot.
• per-once: executes only the first time that the operating system boots,
even if there’s changes in the instance ID. This behaviour is similar to the
trust from the per-instance frequency. Resources set to this frequency can
be executed again by manually cleaning the cache and rebooting the op-
erating system.
To manually clear the cloud-init cache, just execute the command:
cloud-init clean
5.3. Known datasources
The cloud-init have integrations for the biggest cloud orchestrators, such as
Azure, Google Compute Engine, OpenStack e CloudStack (full list can be found
in known-sources). Each of them operates in a unique way and have their own
settings. For CloudStack, as an example, it’s needed that the virtual machine is
within a network that have a virtual router, because it’s from it that cloud-init
will retrive the user information and metadata.
5.4. Modules
The cloud-init have many functionalities, such as password management,
SSH key injection and user metadata injection ( full list can be found in mod-
ules). If the user needs any functionality not included in cloud-init, still they
can create customized scripts (more information in the section 5.1). For each
script is necessary to inform, on its first line, which program will execute it. For
example:
#!/bin/bash
It’s necessary to pay attention to the execution frequency of the applied
modules. Modules like ssh and set-passwords executes, by default, at per-instance
271
frequency, i.e., will only be executed during the first boot of the operating sys-
tem. If necessary to change this frequency, cloud-init allows to overwrite it in
the following way:
...
cloud_init_modules:
...
- update_etc_hosts
- ca-certs
- rsyslog
- users-groups
- [ ssh, always ]
...
5.5. Creating a customized image
It’s possible to create a customized image via CloudStack by following these
steps:
• Create a VM from an ISO, as described in the sections 2.10 and ??.
• With the operating system installed and setup, install the cloud-init pack-
age.
• By default, cloud-init tries to find the data from all datasources. If it’s un-
able through one, it tries with another one. However, it’s possible to limit
the datasources that the cloud-init will utilize. As this image will be des-
ignated to the CloudStack, the datasources will be limited to it. For this,
in the directory /et c/cloud/clo ud.cfg.d/, create the file 99_cloudstack.cfg
with the following data:
datasource:
CloudStack: {}
None: {}
datasource_list: [CloudStack, None]
This setting will limit the datasources to CloudStack and None. If the data
aren’t found by CloudStack, no search for them will be made from any
other. Other settings may also be created in the same directory and cloud-
init will read them in alphabetical order, however, by default, they will be
defined in the file /etc/cloud/cloud.cfg.
272
• In the file / etc/c loud/ cloud .cfg, under the users section, users to be cre-
ated can be defined. By default, there’s already an item that refers to the
default user, in the section system_info. The default user values can be
changed to match the desired user.
• In the file /etc/cloud/cloud.cfg, when installing the cloud-init package, sev-
eral modules come pre-defined. It’s possible to change their properties
as well as add or remove modules whenever needed. More information
about modules in section 5.4.
To change the host name it’s necessary to use the modules set_hostname,
update_hostname and update_etc_hosts. To inject SSH keys it’s necessary
to use the ssh module. To update the default user password it’s necessary
to use the module set-passwords.
Here’s an example of module setup and all the steps for this:
# The modules that run in the 'init' stage
cloud_init_modules:
- migrator
- seed_random
- bootcmd
- write-files
- growpart
- resizefs
- disk_setup
- mounts
- set_hostname
- update_hostname
- update_etc_hosts
- ca-certs
- rsyslog
- users-groups
- [ssh, always]
# The modules that run in the 'config' stage
cloud_config_modules:
# Emit the cloud config ready event
# this can be used by upstart jobs for 'start on cloud-config'.
- emit_upstart
- snap
- ssh-import-id
- locale
- [set-passwords, always]
- grub-dpkg
- apt-pipelining
- apt-configure
- ubuntu-advantage
- ntp
- timezone
- disable-ec2-metadata
273
- runcmd
- byobu
# The modules that run in the 'final' stage
cloud_final_modules:
- package-update-upgrade-install
- fan
- landscape
- lxd
- ubuntu-drivers
- chef
- reset_rmc
- refresh_rmc_and_interface
- rightscale_userdata
- scripts-per-boot
- ssh-authkey-fingerprints
- keys-to-console
- phone-home
- power-state-change
• When executing the ssh module, cloud-init also will generate new public
keys for the host. If the user save their fingerprints, they might face the
following message and become unable to connect until the issue is fixed:
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the ECDSA key sent by the remote host is
...
To prevent this situation, that will occur every time the user reboots the
VM, it’s possible to add settings to the cloud-init so that it won’t generate
new public keys when executing the ssh module. For that, a file named
49_hostkeys.cfg must be created at the /etc/clou d/cloud.cfg.d/ directory,
containing the following content:
ssh_deletekeys: false
• For a quick validation of the cloud-init settings, it’s possible to turn off the
virtual machine, change its name in CloudStack and turn it on again. As
soon as it starts, its hostname should match the name defined on Cloud-
Stack.
274
• With the settings validated, turn off the virtual machine and create a tem-
plate based in the volume, as described in section 2.10. The template will
be with cloud-init setupped and ready for use.
5.6. Cloud-init in Windows
The cloud-init is a solution developed for Linux, however there’s an equiv-
alent software for Windows called cloudbase-init. It also have an integration
with the CloudStack, with support, but not limited to alteração de hostname,
alteração de senha and injeção de chaves SSH.
5.7. Troubleshooting cloud-init
If an error occurs during th execution of cloud-init modules, it’s possible to
begin troubleshooting through the logs recorded in the files /var/log/cloud-ini
t*.
5.8. Force password change during first login
When cloud-init injects the default user password, it’s interesting, for secu-
rity sake, force its change during the first login.
For that, it’s necessary to create a script in the VM that have cloud-init in-
stalled. Just follow these steps:
1. Access the VM containing cloud-init;
2. create the script file:
sudo touch /var/lib/cloud/scripts/per-instance/reset_password.sh
3. add a Linux command to expire password within the created file (in <d ef
ault-user> the default user from the VM must be informed):
#!/bin/bash
echo "Default user password expired to force password change during first access"
passwd --e <default-user>
4. change the access file so that it become an executable:
sudo chmod +x /var/lib/cloud/scripts/per-instance/reset_password.sh
275
After these steps, just create a template based in this VM, as in section 2.10.
Then, all VMs created from this template will force the user to change the pass-
word during their first login.
5.9. Self incrementing disk size during boot
The following process were tested by SC Clouds team with the Ubuntu 20.04
LTS operating system, Cloud.init version 21.4, LVM version 2.03.07. We don’t
guarantee that it fully functions for other systems, libraries and compo-
nents. Use this section only as an example.
The automatic disk resizing during boot it’s possible through a script that will
be executed per-boot by Cloud.init. For this it’s required that the last partition of
th device is set as a volume group (VG) and with a logical volume (LV) setupped
in this VG.
1. Access the VM containing cloud-init;
2. Create a script file:
sudo touch /var/lib/cloud/scripts/per-boot/resizelvm.sh
3. The following script expands the LV with the whole available block at the
end of the disk, assuming that:
(a) The device is /dev/vda
(b) The partition containing the VG is the 3
(c) The LV path is /dev/vg0/lv0
#!/bin/bash
DEVICE="/dev/vda"
LVPARTITION="3"
LVPATH="/dev/vg0/lv0"
growpart "$DEVICE" "$LVPARTITION"
pvresize "$DEVICE""$LVPARTITION"
lvextend -l +100%Free "$LVPATH"
resize2fs "$LVPATH"
4. Grant execution permission for this script:
sudo chmod +x /var/lib/cloud/scripts/per-boot/resizelvm.sh
276
With that, the instance will verify if there’s enough available space to expand
the LVM during the boot process and, if enough, the expansion will be per-
formed allocating available blocks of the partition pre-configured in the script.
5.10. Interactions with user-data
It’s important to keep in mind that the interaction between cloud-init and the
user-data script from the VM. The user-data executes after cloud-init. There-
fore, if cloud-init defines the password for the users:
chpasswd:
expire: false
users:
- name: ubuntu
password: cloudinit
type: text
- name: test
password: cloudinit
type: text
And the user-data script also does that:
#!/bin/bash
echo "ubuntu:userdata" > senhas
cat senhas | chpasswd
rm senhas
The settings from cloud-init will be overwritten by those from user-data. In
the example, the password for the user ubuntu will be us erdata, and for the
user test will be cloudinit.
277
6. Conclusion
This document addressed, in general, the main concepts and recurring doubts
about the Apache CloudStack and auxiliary tools. Both are vast and complex,
therefore it’s not possible to adress them in their entirety, however, in case of
any doubts or suggestions for improvement in this documentation, a issue can
be created on GitLab.
278
Appendices
279
Appendix A. Terminology
• Infrastructure-as-a-Service:
Infrastructure-as-a-Service, also known as IaaS, consists in the act of of-
fering computational resources, such as processing, storage and network
access, on demand, usually related to a pricing related to the usage of
those resources.
• Hypervisor:
Software responsible for the creation and execution of virtual machines
(VMs). A hypervisor allows that a host computer provides support to sev-
eral guest VMs, virtually sharing its resources, such as memory, storage
and processing, and thus allowing a better usage for the available re-
sources.
• VM:
Acronym for Virtual Machine. It’s a software that emulates a real com-
puter. Also called as guest, it’s created on another computer, known as
the host, and uses part of their resources. The advantages of using VMs
are:
– Allow using different operating systems in a single computer.
– Extremely easy to manage and maintain.
– VMs can be easily created or replicated with an operating system pre-
viously installed and configured.
– VMs can also be easily migrated from one host to another without
data loss.
• Virtual network card:
Commonly refferred to as NIC, VNIC or VIF. It’s a software that fulfills the
role of a network card is a virtual system.
280
