Streamlining My GCP SSH Workflow: Using SSH Configuration Tricks

How I simplified SSH access to GCP instances, increased security, and decreased points of failure.

3 min

I often recreate virtual machines (VMs) in Google Cloud Platform (GCP), which means their connection information changes frequently. That leaves stale entries in known_hosts and creates more DNS work as the number of servers grows. Using GCP's Identity-Aware Proxy (IAP) also requires the gcloud CLI. So instead of dealing with this complexity each time, I used my past exprience with SSH configuration customization to handle those moving parts.

Editor's Note: At the time I made heavy use of GCP, however now I am using other hyperscalers. This post is kept up for historical purposes.

The challenge

There were several issues I needed to solve:

  1. Frequent VM recreation leading to connection information changes
  2. Managing stale known_hosts entries
  3. DNS management for numerous servers
  4. The necessity of using IAP for secure access
  5. The complexity of the gcloud compute ssh command

Requirements

The setup needs:

  • The gcloud CLI installed locally
  • nc (netcat) installed on the remote servers to proxy the connection from IAP to the VM

gcloud compute ssh wraps ssh to handle authentication and IP resolution. It becomes unwieldy with other SSH-based tools such as Ansible and rsync because each tool needs the command passed through to it. I had previously used Vault to make SSH run a command for each connection, so I tried the same approach here. A Stack Overflow answer gave me a starting point that I adapted for this setup.

I use GCP's Identity-Aware Proxy to tunnel connections without exposing SSH to the public or assigning public IPs to each VM. The gcloud CLI provides this through the --tunnel-through-iap flag.

SSH configuration

I added this block to my SSH configuration:

Host *.gcloud
    ProxyCommand bash -c 'IFS=. read -r server zone project _ <<< "${1}"; gcloud compute --project "$project" ssh --zone "$zone" --ssh-key-file ~/.ssh/gcp_key --tunnel-through-iap "${2}@$server" --command="nc 0.0.0.0 22"' _ %h %r
    IdentityFile ~/.ssh/gcp_key
    # GCP handles the known hosts, and hostkey checking separately, so we can ignore them here
    StrictHostKeyChecking no
    UserKnownHostsFile=/dev/null

This configuration takes a hostname I provide (e.g., server1.us-west1-b.project1-id.gcloud) and converts it into variables that are then passed to the gcloud CLI.

I also wanted to forward ports from inside GCP's network to my local machine. For a particular host, define a more specific Host block and add its options there. SSH will also apply the settings from the wildcard block.

Host server1.us-west1-b.project1-id.gcloud
    LocalForward 3306 10.20.30.40:3306

# Host *.gcloud...

Troubleshooting

If this setup does not connect, make sure you've added the IAP IP addresses to your GCP network ingress rules. You can find the list in the GCP documentation. You may also need to sign in to the gcloud CLI.

This setup has saved me a lot of time. I could manage DNS entries when creating servers with infrastructure as code, but that would add another piece of configuration that can fail. This approach also keeps the VMs' SSH ports off the public internet.