SSH Keys and Jump Servers: A Beginner's Guide to Doing It Right

The Problem in One Sentence

When you copy your private SSH key onto a jump server and CVE-2026-31431 — or any local privilege escalation — runs on that server, an attacker can read every file the compromised account can access, including your key. If the key has no passphrase, they immediately have everything that key unlocks.

This guide explains why that is dangerous, what a safe setup looks like, and how to connect through a jump server without ever putting your private key on it.

What Is an SSH Key Pair?

SSH authentication using keys works differently from passwords. Instead of a secret you type, you generate two mathematically linked files:

• Private key — a secret file that proves your identity. Think of it as a master key. It should exist in exactly one place: the machine you work from.
• Public key — a file you can share freely. You place it on any server you want to log into. It acts like a lock that only your private key can open.

When you connect to a server, SSH uses these two files together to authenticate you without transmitting your private key over the network. The server has the public key (the lock); you have the private key (the key). The private key never leaves your machine — it just proves it exists by answering a cryptographic challenge.

Generating a Key Pair

ssh-keygen -t ed25519 -C "[email protected]"

When prompted for a passphrase, enter one. This is covered in detail below. Your keys are saved to:

~/.ssh/id_ed25519       ← private key — keep this on THIS machine only
~/.ssh/id_ed25519.pub   ← public key — safe to copy anywhere

The Golden Rule: Your Private Key Never Leaves the Machine That Generated It

This is the most important concept in SSH key management and the one most often violated by beginners.

If your private key exists on two machines, it is twice as exposed. Every machine holding your private key is a machine that, if compromised, gives an attacker your key — and from there, access to everything that key unlocks.

The consequences are asymmetric: putting your key on a server that gets exploited is not just losing access to that server. It is losing access to every server your key can reach.

The rule, stated plainly:

• Generate the key on the machine you work from (your laptop, your workstation).
• Copy the public key (.pub) to servers.
• Never copy the private key anywhere.

What Is a Jump Server?

A jump server (also called a bastion host) is a single machine that acts as a controlled gateway into a private network. Instead of exposing all your internal servers to the internet, you expose only the jump server — with strong access controls, logging, and monitoring on it. To reach an internal server, you first connect to the jump server, then from there to your destination.

Your laptop → jump server (public-facing) → internal server (private network)

This architecture is sound. The mistake is in how people implement the "two-hop" connection.

The Wrong Way: Copying Your Key to the Jump Server

The intuitive but dangerous approach is:

1. Copy your private key to the jump server.
2. SSH to the jump server.
3. SSH from the jump server to the internal server.

This is how many people first set up jump server access, and it is exactly what you should not do. Your private key now lives on the jump server. The jump server is, by definition, the most exposed machine in your infrastructure — it faces the internet and is accessible to anyone who can reach it. It is the most likely target for attack.

When CVE-2026-31431 disclosed a local privilege escalation in the Linux kernel, the practical implication for anyone with keys on their jump server was immediate: an attacker who could run code as any unprivileged user on that server — something the Copy Fail exploit permits — could read your private key from ~/.ssh/ and use it against every server behind the bastion.

A passphrase-protected key buys time (the attacker has the encrypted key but must crack the passphrase). A key with no passphrase gives instant access to every host it unlocks.

The Right Way: ProxyJump

SSH has a built-in mechanism called ProxyJump (available since OpenSSH 7.3, released 2016) that tunnels your connection through a jump server without your private key ever touching it.

Here is how it works:

1. Your machine opens an SSH connection to the jump server.
2. Through that connection, your machine opens a second SSH connection to the internal server.
3. Both connections originate from your machine. Your private key is used only on your machine. The jump server never sees it.

From the command line

ssh -J [email protected] [email protected]

The -J flag specifies the jump host. SSH handles the tunneling transparently.

From ~/.ssh/config (recommended)

A config entry makes this seamless — no flags to remember:

Host jumpserver
    HostName jumpserver.example.com
    User your-username
    IdentityFile ~/.ssh/id_ed25519

Host internal
    HostName 10.0.0.50
    User your-username
    IdentityFile ~/.ssh/id_ed25519
    ProxyJump jumpserver

With this config, connecting to internal is just:

ssh internal

SSH automatically tunnels through jumpserver first. Your private key stays on your laptop. The jump server sees an incoming connection and an outgoing connection — it never sees your key.

Passphrases: Your Last Line of Defense

A passphrase encrypts your private key file on disk. Without the passphrase, the file is useless to anyone who obtains it.

Without a passphrase:

• An attacker who gets the file has your key immediately.
• A local privilege escalation on any machine holding the key means instant compromise of all downstream hosts.

With a passphrase:

• The file alone is useless. The attacker must also crack the passphrase.
• Strong passphrases (random words, 20+ characters) are infeasible to crack.
• The passphrase is never transmitted — it only ever unlocks the key locally.

Using ssh-agent So You Only Type It Once

A passphrase on every connection would be tedious. ssh-agent solves this: it holds your decrypted key in memory for your session so you type the passphrase once, then it handles authentication silently for the rest of the session.

eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_ed25519

You will be prompted for your passphrase once. SSH connections during that session use the agent automatically.

On macOS the system SSH agent starts automatically. On Linux most desktop environments start one too. On Windows, enable the OpenSSH Authentication Agent service.

The ForwardAgent Trap

AgentForwarding is a different feature that is often confused with ProxyJump, and it is dangerous in a way that ProxyJump is not.

When ForwardAgent yes is set, the remote server gets a socket that can ask your local SSH agent to perform signing operations. This means code running on the remote server — including any process the server operator runs — can use your key to authenticate to other servers, without ever seeing the key itself.

If the jump server is compromised, ForwardAgent gives an attacker a live, functioning credential. There is no key file to steal — they just use the forwarded socket.

The fix is simple: do not use ForwardAgent. Use ProxyJump instead. They accomplish the same goal (reaching servers behind a bastion) with completely different security properties.

If you have ForwardAgent yes in your SSH config anywhere, remove it unless you have a specific reason and fully understand the implications.

Putting It All Together

A secure jump server setup looks like this:

1. Generate your key on your workstation with ssh-keygen -t ed25519 and set a strong passphrase.
2. Copy the public key to the jump server and internal servers with ssh-copy-id.
3. Configure ProxyJump in ~/.ssh/config so SSH tunnels automatically.
4. Never copy the private key (id_ed25519, no .pub) anywhere. If you find a copy of it on a server, delete it.
5. Do not use ForwardAgent.
6. Use ssh-agent locally so you type your passphrase once per session, not once per connection.

If your jump server is ever compromised — by a kernel LPE, a misconfigured service, or anything else — an attacker who reads your ~/.ssh/ directory on that server finds only your public key (safe to expose) and nothing else. The private key that could unlock your internal network never left your machine.

Quick Reference

# Generate a key (do this on your workstation, not a server)
ssh-keygen -t ed25519

# Copy the PUBLIC key to a server
ssh-copy-id -i ~/.ssh/id_ed25519.pub [email protected]

# Connect through a jump server (one-off)
ssh -J [email protected] [email protected]

# ~/.ssh/config for permanent ProxyJump setup
Host internal
    HostName 10.0.0.50
    User yourname
    ProxyJump jumpserver

# Start ssh-agent and add your key (type passphrase once)
eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_ed25519