cloud,

Google Cloud Professional Cloud Architect Certification - 2 Core

Cui Cui Follow May 17, 2023 · 12 mins read
Google Cloud Professional Cloud Architect Certification - 2 Core
Share this

“I paint with a brayer and press. A color lover with monochrome moods”. -Kathleen DeMeo

A. [Course] Essential Google Cloud Infrastructure: Core Services

A.1. Infrastructure Preview

a sophisticate deployment in minutes

  • Jenkins CI deployment
  • Jenkins UI management
  • VM SSH administration

Deployment Manager (GCP Marketplace)

` Deployment Manager is a Google Cloud service that uses templates written in a combination of YAML, python, and Jinja2 to automate the allocation of Google Cloud resources and perform setup tasks. Behind the scenes a virtual machine has been created. A startup script was used to install and configure software, and network Firewall Rules were created to allow traffic to the service. `

Virtual Private Cloud (VPC)

  • How to calculate IP range? 0.0.0.0/31 (32) 0.0.0.0/23 (24) 0.0.0.0/15 (16) 0.0.0.0/7 (8)
  • Internal DNS resolver: 169.254.169.254
  • DHCP: domain/ip mapping lookup
  • Cloud DNS - Public domain and External IP
# create vpc networks
gcloud compute networks create privatenet --subnet-mode=custom
# create vpc networks subnets
gcloud compute networks subnets create privatesubnet-us --network=privatenet --region=us-central1 --range=172.16.0.0/24
gcloud compute networks subnets create privatesubnet-eu --network=privatenet --region=europe-west1 --range=172.20.0.0/20
# list networks
gcloud compute networks list
# list networks subnets
gcloud compute networks subnets list --sort-by=NETWORK


# create firewall rules
gcloud compute firewall-rules create privatenet-allow-icmp-ssh-rdp --direction=INGRESS --priority=1000 --network=privatenet --action=ALLOW --rules=icmp,tcp:22,tcp:3389 --source-ranges=0.0.0.0/0
# list 
gcloud compute firewall-rules list --sort-by=NETWORK
# create vm with custom networks
gcloud compute instances create privatenet-us-vm --zone=us-central1-c --machine-type=f1-micro --subnet=privatesubnet-us --image-family=debian-10 --image-project=debian-cloud --boot-disk-size=10GB --boot-disk-type=pd-standard --boot-disk-device-name=privatenet-us-vm
# list vms
gcloud compute instances list --sort-by=ZONE


# SSH vm through Cloud shell
gcloud compute ssh vm-internal --zone us-central1-c --tunnel-through-iap

# format and mount disk to a folder
sudo mkdir -p /home/minecraft
sudo mkfs.ext4 -F -E lazy_itable_init=0,lazy_journal_init=0,discard /dev/disk/by-id/google-minecraft-disk
sudo mount -o discard,defaults /dev/disk/by-id/google-minecraft-disk /home/minecraft

Compute Engine - Virtual Machines

A.2. Knowledge notes

  • external IP is just mapping to internal IP, so VM doesn’t know its public IP. ephemeral public IP, or reserved static public IP.
  • internal IP access GCP API, by enabling Private Google Access on the subnet configure;
  • SSH firewall rule IP range : 35.235.240.0/20 see Using IAP for TCP forwarding.
  • Role IAP Secured Tunnel User is default to VM instance owner, and can assign to other users through IAM.

  • Cloud NAT is only outbound gateway:

    The Cloud NAT gateway implements outbound NAT, but not inbound NAT. In other words, hosts outside of your VPC network can only respond to connections initiated by your instances; they cannot initiate their own, new connections to your instances via NAT.

  • Preemptible
  • Sole-tenant host
  • Boot disk, Disable delete when vm is deleted, so that keep disk data for next reboot
  • Persistent disk,
  • Local SSD disks, are physically attached to VMs
  • RAM disk tmpfs, faster than disk, slower than memory.
  • 16 disks for shared core, 128 disks can be attached for others.

A.3. Reference

B. [Quest] GCP Create and Manage Cloud Resources

gcloud config set compute/zone gcloud config set compute/region

B.1. GKS

gcloud container clusters create --machine-type=e2-medium --zone= lab-cluster 

gcloud container clusters get-credentials lab-cluster

kubectl create deployment hello-server --image=gcr.io/google-samples/hello-app:1.0

kubectl expose deployment hello-server --type=LoadBalancer --port 8080

kubectl get service

gcloud container clusters delete lab-cluster

B.2. Http LoadBalancer

Web service 3 instances

gcloud compute instances create www1 \
    --zone= \
    --tags=network-lb-tag \
    --machine-type=e2-medium \
    --image-family=debian-11 \
    --image-project=debian-cloud \
    --metadata=startup-script='#!/bin/bash
      apt-get update
      apt-get install apache2 -y
      service apache2 restart
      echo "<h3>Web Server: www1</h3>" | tee /var/www/html/index.html'


gcloud compute instances create www2 \
    --zone= \
    --tags=network-lb-tag \
    --machine-type=e2-medium \
    --image-family=debian-11 \
    --image-project=debian-cloud \
    --metadata=startup-script='#!/bin/bash
      apt-get update
      apt-get install apache2 -y
      service apache2 restart
      echo "<h3>Web Server: www2</h3>" | tee /var/www/html/index.html'


gcloud compute instances create www3 \
    --zone= \
    --tags=network-lb-tag \
    --machine-type=e2-medium \
    --image-family=debian-11 \
    --image-project=debian-cloud \
    --metadata=startup-script='#!/bin/bash
      apt-get update
      apt-get install apache2 -y
      service apache2 restart
      echo "<h3>Web Server: www3</h3>" | tee /var/www/html/index.html'

	gcloud compute instances list


# Firewall Rule
gcloud compute firewall-rules create www-firewall-network-lb --target-tags network-lb-tag --allow tcp:80

# IP for load balancer
gcloud compute addresses create network-lb-ip-1 --region 

# Health check for LB
gcloud compute http-health-checks create basic-check

# Target pool for health check
gcloud compute target-pools create www-pool --region --http-health-check basic-check

# Add instances to target pool
gcloud compute target-pools add-instances www-pool --instances www1,www2,www3

# Add Forwarding rule between LB and target pool
gcloud compute forwarding-rules create www-rule \
    --region   \
    --ports 80 \
    --address network-lb-ip-1 \
    --target-pool www-pool

# Verify now
gcloud compute forwarding-rules describe www-rule --region 

IPADDRESS=$(gcloud compute forwarding-rules describe www-rule --region --format="json" | jq -r .IPAddress)

while true; do curl -m1 $IPADDRESS; done

B.3. Global LoadBalancing with URL mapping

# Load Balancer template

gcloud compute instance-templates create lb-backend-template \
   --region= \
   --network=default \
   --subnet=default \
   --tags=allow-health-check \
   --machine-type=e2-medium \
   --image-family=debian-11 \
   --image-project=debian-cloud \
   --metadata=startup-script='#!/bin/bash
     apt-get update
     apt-get install apache2 -y
     a2ensite default-ssl
     a2enmod ssl
     vm_hostname="$(curl -H "Metadata-Flavor:Google" \
     http://169.254.169.254/computeMetadata/v1/instance/name)"
     echo "Page served from: $vm_hostname" | \
     tee /var/www/html/index.html
     systemctl restart apache2'

# Managed instance groups based on template
gcloud compute instance-groups managed create lb-backend-group --template=lb-backend-template --size=2 --zone= 

# Firewall rule
gcloud compute firewall-rules create fw-allow-health-check \
  --network=default \
  --action=allow \
  --direction=ingress \
  --source-ranges=130.211.0.0/22,35.191.0.0/16 \
  --target-tags=allow-health-check \
  --rules=tcp:80

# public IP for LB
gcloud compute addresses create lb-ipv4-1 --ip-version=IPV4 --global
gcloud compute addresses describe lb-ipv4-1 --format="get(address)" --global

# health check for LB
gcloud compute health-checks create http http-basic-check --port 80

# backend service
gcloud compute backend-services create web-backend-service \
  --protocol=HTTP \
  --port-name=http \
  --health-checks=http-basic-check \
  --global

# add instance group as backend service
gcloud compute backend-services add-backend web-backend-service \
  --instance-group=lb-backend-group \
  --instance-group-zone= \
  --global

# add url mapping between request path and backend services
gcloud compute url-maps create web-map-http --default-service web-backend-service

# http proxy to route requests to url map
gcloud compute target-http-proxies create http-lb-proxy --url-map web-map-http

# global forwarding rule between incoming requests and http proxy
gcloud compute forwarding-rules create http-content-rule \
    --address=lb-ipv4-1\
    --global \
    --target-http-proxy=http-lb-proxy \
    --ports=80

B. [Quest] Deploy and Manage Cloud Environments with Google Cloud

B.1. Cloud IAM

B.2. Introduction to SQL, for BigQuery and Cloud SQL

Keywords:

  • SELECT,
  • FROM,
  • WHERE,
  • GROUP BY, Your results are a list of unique (non-duplicate) column values.
  • COUNT,
  • AS,
  • ORDER BY
  • UNION, This keyword combines the output of two or more SELECT queries into a result-set.

BigQuery

BigQuery is a fully-managed petabyte-scale data warehouse that runs on the Google Cloud.

Data analysts and data scientists can quickly query and filter large datasets, aggregate results, and perform complex operations without having to worry about setting up and managing servers.

It comes in the form of a command line tool (pre installed in cloudshell) or a web console—both ready for managing and querying data housed in Google Cloud projects.

BigQuery -> projects -> datasets -> tables

Cloud SQL

Cloud SQL is a fully-managed database service that makes it easy to set up, maintain, manage, and administer your relational PostgreSQL and MySQL databases in the cloud.

There are two formats of data accepted by Cloud SQL: dump files (.sql) or CSV files (.csv).

You will learn how to export subsets of the cycle_hire table into CSV files and upload them to Cloud Storage as an intermediate location.

  • Cloud Shell CLIs ```Shell PROJECT_ID=qwiklabs-gcp-01-139137c0accc gcloud config set project $PROJECT_ID

gcloud auth login –no-launch-browser

gcloud sql connect qwiklabs-demo –user=root –quiet

CREATE DATABASE bike; USE bike; CREATE TABLE london1 (start_station_name VARCHAR(255), num INT);

USE bike; CREATE TABLE london2 (end_station_name VARCHAR(255), num INT);

SELECT * FROM london1; SELECT * FROM london2;

DELETE FROM london1 WHERE num=0; DELETE FROM london2 WHERE num=0;

INSERT INTO london1 (start_station_name, num) VALUES (“test destination”, 1);

SELECT start_station_name AS top_stations, num FROM london1 WHERE num>100000 UNION SELECT end_station_name, num FROM london2 WHERE num>100000 ORDER BY top_stations DESC;



### B.3. Multiple VPC Networks

#### Create VPC networks and subnets

```bash
gcloud compute networks create managementnet --subnet-mode=custom --mtu=1460 --bgp-routing-mode=regional

gcloud compute networks subnets create managementsubnet-us --range=10.130.0.0/20 --stack-type=IPV4_ONLY --network=managementnet --region=us-east1



gcloud compute networks create privatenet --subnet-mode=custom --mtu=1460 --bgp-routing-mode=regional

gcloud compute networks subnets create privatesubnet-us --range=172.16.0.0/24 --stack-type=IPV4_ONLY --network=privatenet --region=us-east1

gcloud compute networks subnets create privatesubnet-eu --range=172.20.0.0/20 --stack-type=IPV4_ONLY --network=privatenet --region=europe-west4 


gcloud compute networks list

gcloud compute networks subnets list --sort-by=NETWORK

Create firewall rules

gcloud compute firewall-rules create managementnet-allow-icmp-ssh-rdp --direction=INGRESS --priority=1000 --network=managementnet --action=ALLOW --rules=tcp:22,tcp:3389,icmp --source-ranges=0.0.0.0/0

gcloud compute firewall-rules create privatenet-allow-icmp-ssh-rdp --direction=INGRESS --priority=1000 --network=privatenet --action=ALLOW --rules=icmp,tcp:22,tcp:3389 --source-ranges=0.0.0.0/0

gcloud compute firewall-rules list --sort-by=NETWORK

Create VM instances

gcloud compute instances create managementnet-us-vm --project=qwiklabs-gcp-00-d2691a0a4e47 --zone=us-east1-c --machine-type=e2-micro --network-interface=network-tier=PREMIUM,subnet=managementsubnet-us --metadata=enable-oslogin=true --maintenance-policy=MIGRATE --provisioning-model=STANDARD --service-account=574892642852-compute@developer.gserviceaccount.com --scopes=https://www.googleapis.com/auth/devstorage.read_only,https://www.googleapis.com/auth/logging.write,https://www.googleapis.com/auth/monitoring.write,https://www.googleapis.com/auth/servicecontrol,https://www.googleapis.com/auth/service.management.readonly,https://www.googleapis.com/auth/trace.append --create-disk=auto-delete=yes,boot=yes,device-name=managementnet-us-vm,image=projects/debian-cloud/global/images/debian-11-bullseye-v20220920,mode=rw,size=10,type=projects/qwiklabs-gcp-00-d2691a0a4e47/zones/us-east1-c/diskTypes/pd-balanced --no-shielded-secure-boot --shielded-vtpm --shielded-integrity-monitoring --reservation-affinity=any

gcloud compute instances create privatenet-us-vm --zone="us-east1-c" --machine-type=e2-micro --subnet=privatesubnet-us

gcloud compute instances list --sort-by=ZONE


gcloud compute ssh --zone "us-east1-c" "mynet-us-vm"  --project "qwiklabs-gcp-00-d2691a0a4e47"
gcloud compute ssh --zone "us-east1-c" "managementnet-us-vm"  --project "qwiklabs-gcp-00-d2691a0a4e47"

// external_ip

ping -c 3 35.204.82.219
ping -c 3 35.196.96.138
ping -c 3 35.243.178.120

// internal_ip
ping -c 3 10.164.0.2
ping -c 3 10.130.0.2
ping -c 3 172.16.0.2

Create VM instances with multiple network interfaces

gcloud compute instances create vm-appliance --project=qwiklabs-gcp-00-d2691a0a4e47 --zone=us-east1-c --machine-type=e2-standard-4 --network-interface=network-tier=PREMIUM,subnet=privatesubnet-us --network-interface=network-tier=PREMIUM,subnet=managementsubnet-us --network-interface=network-tier=PREMIUM,subnet=mynetwork --metadata=enable-oslogin=true --maintenance-policy=MIGRATE --provisioning-model=STANDARD --service-account=574892642852-compute@developer.gserviceaccount.com --scopes=https://www.googleapis.com/auth/devstorage.read_only,https://www.googleapis.com/auth/logging.write,https://www.googleapis.com/auth/monitoring.write,https://www.googleapis.com/auth/servicecontrol,https://www.googleapis.com/auth/service.management.readonly,https://www.googleapis.com/auth/trace.append --create-disk=auto-delete=yes,boot=yes,device-name=vm-appliance,image=projects/debian-cloud/global/images/debian-11-bullseye-v20220920,mode=rw,size=10,type=projects/qwiklabs-gcp-00-d2691a0a4e47/zones/us-east1-c/diskTypes/pd-balanced --no-shielded-secure-boot --shielded-vtpm --shielded-integrity-monitoring --reservation-affinity=any


gcloud compute ssh --zone "us-east1-c" "vm-appliance"  --project "qwiklabs-gcp-00-d2691a0a4e47"

sudo ifconfig

ping -c 3 172.16.0.2
ping -c 3 35.243.178.120
ping -c 3 privatenet-us-vm

ip route

More to learn

B.4. Cloud Monitoring

B.5. Managing Deployments Using K8s Engine

B.6. Set Up and Configure a Cloud Environment in Google Cloud: Challenge Lab

CREATE DATABASE wordpress;
CREATE USER "wp_user"@"%" IDENTIFIED BY "stormwind_rules";
GRANT ALL PRIVILEGES ON wordpress.* TO "wp_user"@"%";
FLUSH PRIVILEGES;

gcloud container clusters create griffin-dev --zone=us-east1-b 

gcloud container --project "qwiklabs-gcp-01-f146c3bd35a0" clusters create-auto "griffin-dev" --zone "us-east1-b" --release-channel "regular" --network "projects/qwiklabs-gcp-01-f146c3bd35a0/global/networks/griffin-dev-vpc" --subnetwork "projects/qwiklabs-gcp-01-f146c3bd35a0/regions/us-east1/subnetworks/griffin-dev-wp" --cluster-ipv4-cidr "/17" --services-ipv4-cidr "/22"

gcloud iam service-accounts keys create key.json \
    --iam-account=cloud-sql-proxy@$GOOGLE_CLOUD_PROJECT.iam.gserviceaccount.com
kubectl create secret generic cloudsql-instance-credentials \
    --from-file key.json
Join Newsletter
Get the latest news right in your inbox. We never spam!
Cui
Written by Cui Follow
Hi, I am Z, the coder for cuizhanming.com!
Software Engineering
Google Cloud Professional Cloud Architect Certification - 3 Automation