It's a total pain in the ass to try to have password encrypted gpg or ssh keys in mac. Nothing better that another way to make it even more painful and complicated, so that people will just store plain text keys to not be annoyed.

It's a total pain in the ass to try to have password encrypted gpg or ssh keys in mac.

Who uses password encrypted keys anyway ? No exfiltration protection, and a sitting duck for unlimited automated password guessing attempts.

Pre-Tahoe people used Yubikeys or Secretive. But now this native tool is a better option than Secretive, even if Yubikeys still have their uses for the power-users.

Who uses password encrypted keys anyway ?

Edit: I'm not suggesting an ssh key with a passphrase (or password) is better than what the article suggests; I'm only saying that adding a passphrase (or password) to an ssh key at least buys time to address the situation while the attacker is trying to break the encryption on the stolen key.

I am anti-Mac in every way, but I do use passphrase protected ssh keys so if someone were to get a copy of my ssh key, they would have to be able to break the encryption to use the key. I see a lot of devs using blank passphrases on their ssh keys, smh.

sitting duck for unlimited automated password guessing attempts.

Using a passphrase on your ssh key has nothing to do with whether the ssh service is configured to allow or deny passwords.

whether the ssh service is configured to allow or deny passwords.

Given the consistent use of "password" instead of "passphrase", I think they meant an exfil'ed encrypted key is vulnerable to no-rate-limit bruteforcing, in contrast with hardware-backed keys.

Right, but my context is that devs often use no passsphrase at all. If someone can get a copy, they have instant access to whatever it has access to. They don't need to even break encryption since the key has none if none has been applied. My stance is simply, at least add a passphrase to the key (though some call it a password).

The parent means that an attacker has unlimited attempts at breaking the passphrase on an exfiltrated key. Once the key passphrase is broken, they can log in using the key.

With an ssh agent and time-bounded key expiration one can have very strong password on the key that is convenient to use.

Also password managers like 1password or Bitwarden support ssh-agent protocol so one can have a master password that protects both stored passwords and keys.

How short of a time-bound do you use on your SSH keys?

Whoa, that is pretty cool.

I've been using Secretive for years, and prefer it to all the physical key/card based systems I've tried to get going over the years. I know exactly when my SSH key is used for any operation, because I need to hit a button or do a fingerprint scan. I can keep ssh-agent tunnels to remote boxes so that I can sign git commits remotely without having to worry about a rogue system getting complete access to key ops without me knowing what's going on.

However the Tahoe version of secretive is buggy and frequently locks up on initial key op requests. I don't have the bandwidth to debug it and file a bug report, and honesty I'm not sure I want to relearn all that knowledge of SSH to figure it out.

I think the smart card SSH UX is worse than secretive's, IIRC my past pain, but if it is reliable, worth a shot.

nvm

Secretive is a bit friendlier to set up but I'll probably switch to this anyway so I have one less app on my computer.

Plugging my blog post for how to achieve this on Windows 11:

https://cedwards.xyz/tpm-backed-ssh-keys-on-windows-11/

Is storing ssh-key in tpm possible on Linux?

I’m a bit confused as to why you can export the keys. Can someone explain this?

TFA:

Note that the "private" key here is just a reference to the FIDO credential. It does not contain any secret key material.

If I understand correctly, this means you can't back up the private key, correct? It's in the Secure Enclave, so if you lose your laptop, you also lose the key? Since it looks like export only really exports the public key not the private one?

Probably not the worst thing, you most likely have another way to get into the remote machine, or an admin who can reset you, but still feels like a hole.

Or am I missing something?

ps. It amuses me that my Mac won't let me type Secure Enclave without automatically capitalizing it.

Edit: I understand good security is having multiple keys, I was simply asking if this one can be backed up. OP answered below and is updating their webpage accordingly.

Inability to export the private key is no different from using an YubiKey? You can't "backup" the private key they generate either.

Yeah, that is why you should not [always (depends on your use case)] generate it on a YubiKey.

You need to have:

- an offline master private key backup (air-gapped)

- primary YubiKey (daily use)

- backup YubiKey (locked away)

- revocation certificate (separate storage) (it is your kill-switch)

Having a second YubiKey enrolled is the standard practice.

What people do wrong is:

- They generate directly on YubiKey

- They only use one device

- They do not create a revocation certificate

- They have no offline backups

You generate your GPG keys on a secured system, load the subkeys (not the master because it is not used for daily cryptography) into the YubiKeys, and then remove the secret keys from this system where you generated the keys.

I can understand revocation for GPG, but is revocation ever used for SSH? I could understand it if SSH certificates are used, but honestly I've never encountered an org using SSH's cert system.

You are talking about GPG keys. The featured article only refers to SSH keys. Know the difference.

Yeah, that is why you should not generate it on a YubiKey

No. You should ALWAYS generate on the Yubikey. That's the whole point.

Your backup is one (or more) other keys.

Do you have a good guide/video/write up on this?

I’ve been putting off remaking my GPG and SSH keys using a Yubikey.

This guide [1] mostly follows the practices the previous poster outlined.

[1] https://github.com/drduh/YubiKey-Guide

At https://github.com/drduh/YubiKey-Guide?tab=readme-ov-file#co..., these options are not the most secure one.

  personal-cipher-preferences CHACHA20 AES256 AES192
  personal-digest-preferences BLAKE2B SHA512 SHA384 SHA256
  personal-compress-preferences Uncompressed
  personal-aead-preferences OCB EAX
  default-preference-list BLAKE2B SHA512 SHA384 SHA256 CHACHA20 AES256 AES192 Uncompressed OCB EAX
  cert-digest-algo BLAKE2B
  s2k-digest-algo BLAKE2B
  s2k-cipher-algo CHACHA20
  s2k-count 65011712
  charset utf-8
  no-comments
  no-emit-version
  no-greeting
  keyid-format 0xlong
  list-options show-uid-validity
  verify-options show-uid-validity
  with-fingerprint
  require-cross-certification
  require-secmem
  no-symkey-cache
  armor
  use-agent
  throw-keyids
  weak-digest SHA1 RIPEMD160 MD5
  disable-cipher-algo 3DES CAST5 IDEA BLOWFISH TWOFISH CAMELLIA128 CAMELLIA192 CAMELLIA256
  disable-pubkey-algo RSA1024
  trust-model tofu+pgp
  keyserver hkps://keys.openpgp.org
  keyserver-options no-honor-keyserver-url
  keyserver-options include-revoked
  keyserver-options auto-key-retrieve
  force-mdc
  require-compliance
  compliance de-vs

These are the most secure options (correct me if I am wrong). The only drawback you may encounter is that you need GnuPG 2.3+, and some compatibility tradeoffs.

Does OpenSSH's `sshd` even support GPG key revocation? (Assuming you're talking about using the GnuPG card application of Yubikeys, since the newer "native" FIDO security key implementation of OpenSSH does not support importing existing keys to my knowledge.)

Which makes yubikey impossible to use with geographically distributed backups. You need the backup available at all times for when you want to register with any new service.

This is why you should use a device which allows exporting the seed, like e.g. multi purpose hardware crypto wallets.

Are you talking about SSH or a different setting?

With SSH, you can always share the primary and backup pub keys, even if you don't have the backup key handy.

No I got distracted by the word yubikey. Arguably not the same subject. :)

This is true for passkeys/webauthn/u2f, which is why it’s trash and a completely flawed and not fit for purpose standard (of course the primary purpose is vendor lock-in, not reliable and disaster-proof authentication).

But SSH allows you to export the public key and then you can enroll it on as many hosts as you want without needing access to the private key, so the backup key can remain in a safe, ideally forever as you should never need it.

I agree that it's inconvenient in many cases, but what vendor am I being locked into, exactly? My primary hardware key can be from a completely different vendor than the backup one, so I don't quite buy the conspiracy angle.

There's also no technical obstacle preventing anyone from creating "paired" hardware authenticators that share the same internal root derivation key and can as such authenticate to all services (at least if they don't demand resident credentials) that were registered to any of the keys in the set.

The fact that these keys don't exist on the market (I believe Yubikey looked into them a while ago) more is evidence for the lack of demand, and less for the existence of a cabal, in my view.

Being locked into a set of a handful of vendors who offer "secure" sync (of course, this is not a true PKI and actual key material is being synced, meaning it's only as secure as the syncing protocol and your authentication to it).

Authenticator implementations who allow exports outside of the vendor cartel are threatened to be blacklisted: https://github.com/keepassxreboot/keepassxc/issues/10407

My primary hardware key can be from a completely different vendor than the backup one, so I don't quite buy the conspiracy angle.

The fundamental flaw is that enrolling an authenticator requires it to be present, making a backup strategy much less resilient as it means your secondary device needs to be constantly present and thus exposed to the same local/environmental risks the primary one is (theft/fire/faulty USB port that fries everything plugged in and you only realize after nuking both your keys). It makes an offline backup scenario like with SSH (where you copy a public key and otherwise leave the authenticator out of reach in a safe place) impossible.

Making it hard/impractical to maintain a reliable backup yourself sure makes those proprietary sync-based services attractive, which also doubles as reducing security since key material is being synced and can potentially be extracted (impossible with a true HSM + PKI implementation).

preventing anyone from creating "paired" hardware authenticators

Don't certain types of keys involve writing something to the authenticator, fundamentally preventing this (as the backup authenticator won't get this written value)?

cabal

It doesn't have to be explicit coordinated action like intentionally wanting to prevent people from self-managing passkeys (in fact any hint of it being intentional would be a liability in a potential anti-trust situation, so that's a big no-no); it can be done by simply omitting this scenario, by accident or for "security" purposes, or deprioritizing it to hell. In fact the Credential Migration spec is still a draft and appears quite recent, despite passkeys being heavily promoted for a while: https://fidoalliance.org/specs/cx/cxp-v1.0-wd-20241003.html - you'd think that such as basic feature would be sorted before the push to switch to passkeys no?

In fact you see this exact strategy playing out: https://github.com/keepassxreboot/keepassxc/issues/11363#iss...

For the initial delivery of Credential Exchange, we focused on the most wide use case [emphasis mine]

"Initial" delivery focuses on the most widespread use-case (how convenient it also happens to be the most corporation-friendly use-case), with everything else coming "later", meaning never. I'm sure it'll rot in some Jira backlog as a liability shield so they can promise they did plan for it and just never got around to it, but everyone understands it will never actually get implemented.

How can the "cartel" "blacklist" anyone? The only thing the FIDO alliance can do is not include a vendor's attestation key as trusted in their vendor database, and software solutions aren't on that list to begin with.

The fundamental flaw is that enrolling an authenticator requires it to be present [...]

Yes, but that doesn't mean you can't backup the full authenticator state.

Here's a toy WebAuthN implementation that is backed by a passphrase that you remember or write on a piece of paper which works on many websites supporting passkeys and not enforcing attestation (which is the vast majority, since Apple, Google, 1Password, and Bitwarden all don't support attestation for synchronized credentials a.k.a. passkeys): https://github.com/lxgr/brainchain

Making it hard/impractical to maintain a reliable backup yourself sure makes those proprietary sync-based services attractive

It's also completely open source and can be backed up :) (But again, it's a toy demo – don't use it for anything sensitive!)

How can the "cartel" "blacklist" anyone?

All they have to do is publish a "best practices" statement or some RP certification program mandating attestation to be used (and some PR around how only "certified" RPs are secure) and job done. The only reason they didn't do that yet is that Apple is refusing to play ball and support attestation (but this may change).

The threat was clearly there in the original Github issue, and it's just a temporary inconvenience they can't currently follow through on it.

Yes, but that doesn't mean you can't backup the full authenticator state.

Having the secondary authenticator present in the same vicinity as the primary one exposes it to risks. Having to dump authenticator state at regular intervals now means your backup authenticator must be reachable for writing online, so it can't be a simple "cold storage" backup like a Yubikey in a safe anymore. This also opens up security concerns since you're now dumping and syncing private keys left and right over a network and you lose the peace of mind of using an HSM-backed non-exportable private key where the HSM being unplugged guarantees nobody is currently using your keys.

Seems like a shit ton of complexity and effort to work around a problem OpenSSH elegantly solved 30 years ago.

Here's a toy WebAuthN implementation

Thanks, I will check it out and read up on it. I'd be genuinely happy to move to WebAuthn if I could build my own hardware authenticators that allow the backup one to remain fully offline in a safe, and not have private keys flying around (if I'm doing that, it's not much of an improvement over syncing passwords - except those I can at least type or tell over the phone in an emergency when I need someone else to act on my behalf).

Edit: so it seems like I am mostly right? Only discoverable credentials count as "passkeys", and those generate per-site private keys, meaning offline, cold-storage backups are impossible. I guess I'm sticking to my password manager then since passkeys would provide no improvement in this case.

Check out `man sc_auth`. There's also an exportable variant where the private key is encrypted using the secure enclave as opposed to generated on the secure enclave:

    % sc_auth create-ctk-identity -l ssh-exportable -k p-256 -t bio
    % sc_auth list-ctk-identities
    p-256    A581E5404ED157C4C73FFDBDFC1339E0D873FCAE bio  ssh-exportable ssh-exportable               23.11.26, 19:50 YES  
    % sc_auth export-ctk-identity -h A581E5404ED157C4C73FFDBDFC1339E0D873FCAE -f ssh-exportable.pem
    Enter a password which will be used to protect the exported items:
    Verify password:

You can then re-import it on another device

    % sc_auth import-ctk-identities -f ssh-exportable.pem.p12 -t bio
    Enter PKCS12 file password:

I'll add this to the guide

How is this method any different from encrypting the private key without any secure enclave?

Isn't it just using a password derived key?

The key is stored encrypted with a unique symmetric key that only your secure enclave knows until the point that you export it. It then re-encrypts it with the password.

Until you export it it's just as strong as an enclave-generated one.

Obviously don't keep the exported password encrypted key around and don't use a weak password for export.

The key is stored encrypted with a unique symmetric key that only your secure enclave knows until the point that you export it. It then re-encrypts it with the password.

But what's the security benefit of this compared to having a keyfile? So far as I can tell from the commands you provided, there's no real difference, aside from a hacker having to modify their stealer script slightly.

Why is it more secure: a key file on disk is decrypted into memory every time you enter your passphrase. It means the key is around in plain text in the memory of ssh or ssh-agent. Which means it's extractable by an attacker. An exportable key does all the signing inside the secure enclave and never exposes the decrypted key to OS memory.

The exported key you can keep in a safe for disaster recovery. You shouldn't keep it on your computer of course.

It means the key is around in plain text in the memory of ssh or ssh-agent. Which means it's extractable by an attacker. An exportable key does all the signing inside the secure enclave and never exposes the decrypted key to OS memory.

But malware can just tell the secure enclave to export the key? Yes, they'll have to write new code to do that, but it's not particularly hard (it's 1 line code from your example above), and it's security through obscurity.

The malware would have to prompt for biometric authentication before exporting.

The export operation is guarded by TouchID. So the malware needs to trick you into performing the TouchID gesture.

But yeh the malware only needs to trick you to hit TouchID once. Instead of on each sign operation. So if that's in your threat model don't make the key exportable.

Unlike a TPM and like a YubiKey, you can configure the secure enclave to require presence (via Touch ID) so that a stealer script would be stopped with a prompt.

Is there a way to import an existing (compatible) key and still mark it as non-exportable?

That seems more useful for the SSH key scenario: Generate a key in memory and back it up to offline storage once, and otherwise only use it in a way totally non-exportable by any malware.

This sentence

The exportable private key is encrypted with Elliptic Curve Encryption Standard Variable IVX963 algorithm which is backed by a Secure Enclave key.

makes it sound like exportable keys might inherently not be Secure Enclave resident in Apple's implementation, which would be unfortunate, as anything else can still be accessed by malware with kernel-level privileges.

(GPG, and I believe also PIV, allow importing externally-generated keys without necessarily marking them exportable; they'll just, correctly, lack any attestation statement about having been generated in secure hardware.)

No. There is no way to import an existing key into the Secure Enclave. Only Apple is allowed to do that.

The best you can do is use the SE to decrypt the key and then use the clear text key for encryption/decryption.

This also means that passkeys on macOS/iOS are (at some point) exposed as clear text.

The best you can do is use the SE to decrypt the key and then use the clear text key for encryption/decryption.

AFAIK this is what "secretive" was doing all the time.

I'm pretty sure it creates SE resident keys, which can't be unwrapped by either userspace or the macOS kernel.

Another option is to generate a key and put it on an offline storage, and have a second key only in the SE. This means you'll need to upload two public keys to places to have a backup instead of one, but I think would otherwise achieve the same thing.

The nice thing with this is you can keep your backup public key easily accessible. I try to keep a primary and backup Yubikey on everything important, but you have to physically get the backup Yubikey in order to add it to a site.

You're not really supposed to 'export' keys. Any time you move a key you risk exposing it. The idea of PKI is that only public keys move, the private key stays in one place, ideally never seen.

I've been in the security space for 25 years, and understand the theory of PKI. But I've also been in the ops space for 30 years, and understand that if you don't balance security theory with operational practice, critical business functions can fail.

Ideally yes, the private key is never seen. In reality, it needs to be backed up in a secure place so it can be restored in the event of a failure.

You can use more than one key you know.

Keep the private key you actively use in the secure enclave. The system you actively use is most at risk.

Keep a secondary offline private key as backup. You can generate and store it in a secure location, and never move it around. Airgapped even if you want. You could even use a yubikey or other hardware for the secondary key giving you two hard to export keys.

Distribute pub keys for both of them.

Best of both worlds?

Yeah but if you get a new device, you have to go add its pubkey to every server you ever use. I wish there were an easier way, otherwise it's understandable that people copy privkeys.

There is an easier way: Create a SSH CA, add that to your authorized_keys everywhere, use it to sign the individual public keys.

But now you need to worry about revocation or at least key lifetimes.

I would argue that doing both of those is still less work than maintaining authorized_keys in many places.

If you’re operating at the scale this is too cumbersome to do manually surely you already have a configuration management system in place to automate this no?

There is an easier way, it's called TLS certificates, it's just that SSH decided not to use it for some reason.

Other systems of this nature have figured out long ago that you should be able to have one personal certificate (stored securely in an airgapped environment), from which you'd generate leaf certificates for your devices every year.

It’s important to remember that over time systems develop complexities that can be hard to recover from scratch because by definition air gapped data aren’t ones you are regularly exercising. Here’s an example of this in action from Google’s history

https://google.github.io/building-secure-and-reliable-system...

In what scenario would you prefer to backup an SSH key in favor of generating new SSH keys?

When I have my pub key in the authorized_keys files of many machines, especially machines where I don't control the authorized_keys file.

In this case you can maintain an offline SSH CA and trust that on the remote machines, and then sign yourself leaf certificates against a non-exportable HSM-backed key. In case of loss you just make a new key and sign a new certificate.

Of course this just moves the key management problem somewhere else: now you need to protect the CA key, but that might be easier since you would only need access to it in a disaster recovery scenario if you replaced the laptop or otherwise lost access to your HSM-backed key.

As usual, it all depends on your threat model.

But how do you revoke any compromised certificate if you don't control the remote machines?

Keeping the certificate’s key as non-exportable in the HSM means you do not need to revoke it as it cannot be compromised (not permanently at least), once you’ve regained access to the HSM you can assume the bad guys are out.

Of course the CA key itself is another story, which is why this merely moves the problem elsewhere (however since you only need access to the CA during initial provisioning of a new certificate key, you can better control access to it).

I think you are mixing up concerns. You need a backup key. That doesn't mean you need to backup your key.

Anything (everything?) using SSH authentication supports multiple authentication keys. Have a yubikey in a locked deposit box or something.

Today I make a private/public keypair, and the private key is on my laptop in my encrypted home folder. It also gets backed up to my encrypted offsite backup. That way if my laptop breaks or is stolen, I can restore from backup and be up and running as before.

I was simply asking if that is still possible with this method, nothing more.

And not every service that uses ssh auth allows multiple keys.

It's much safer to export a key one time and import it into a new machine, or store it in a secure backup, than to keep it just hanging out on disk for eternity, and potentially get scooped up by whatever malware happens to run on your machine.

Any malware capable of exfiltrating a file from your home folder is also capable of calling the export command and tricking you into providing biometrics.

Does anybody know if there is something similar for gpg keys? E.g. for commit signing?

That is, natively with the Secure Enclave, not exportable.

Git commits can be signed with ssh keys.

This exists: https://github.com/facebookincubator/sks.

It's a golang library that abstracts usage of ssh keys backed by hardware on all sorts of devices - mostly designed for laptops, but supports Linux, Windows and MacOs

A golang library is cool, but it doesn't give you a working ssh-agent.

I started working on one few years ago: https://github.com/Foxboron/ssh-tpm-agent

How can I get such a key into my iPhone too, so that I can sign emails and file and such with the same private key when I'm on my phone, and my public key is valid for all such operations ? Will iCloud take care of that ? And then I want it all usable from my (multiple) email clients...

Clearly I'm hazy on this stuff. But if I can export the private key from my Mac, is there any use for it on my iPhone, and any way to get it in there ?

Does anybody know why 'p-384-ne' (instead of 'p-256-ne') cannot be used?

Key can be generated, but 'ssh-keygen -w /usr/lib/ssh-keychain.dylib -K -N ""' cannot find the key to export.

Does the hardware only support the NIST curves? Or is that just the example that happens to be given?

Only supports NIST curves and ECDSA yes.

I've heard people make the point before that EdDSA is not great for secure enclaves due to being suspictable to Fault Attacks which could lead to (partial) key extraction

I don't trust the NIST curves: they were generated in a dubious way which has been written about extensively elsewhere (the coefficients for P-256 were generated by hashing the unexplained seed c49d360886e704936a6678e1139d26b7819f7e90). I always avoid them unless I have to use them. It makes me sad when hardware forces me to use them.

I've heard people make the point before that EdDSA is not great for secure enclaves due to being suspictable to Fault Attacks which could lead to (partial) key extraction

Huh, got a link? My understanding is that eddsa is better with respect to side channels in every way, that was part of the intent of it's design. I've worked with crypto hardware which supports it.

I would not trust it personally, specially since the Chip Security Act is looming

https://www.centerforcybersecuritypolicy.org/insights-and-re...

Is there a way to make the lifetime of the key last more than a year?

Why would you want the private key file if you store it within the secure enclave?

Probably for the same reason that OpenSSH's `sk` implementation also still needs a private key file (even for the "resident key" option): You need to be able to point OpenSSH's various tools to something in an identity context, and that something traditionally is a private key file.

The article even mentions that it doesn't contain any sensitive data:

Note that the "private" key here is just a reference to the FIDO credential. It does not contain any secret key material.

It's a slightly different story for non-resident `sk`-backed keys; these actually require the private key file since the hardware authenticator itself is (or at least can be) stateless. (It's still not a security risk if it ever leaks, but it's an availability risk if it's ever lost.)

Not sure if macOS's backing implementation is stateful or stateless (or some unfortunate hybrid of both; i.e., it might just store a stateful wrapped key in some system-level keychain in a way that intransparently breaks if the OS is ever reinstalled, but also doesn't allow querying an intact system for any existing credentials).

This may be the thing to get me to upgrade to MacOS 26.

Ok I wished for this kind of feature for years. I started using a yubikey with an ssh key via gpg ssh-agent in 2018 or 2019. When resident ssh keys came around I switched over to FIDO2 based keys on my yubikey. The main issue with both was the fact that the default ssh setup wasn’t working anymore. One needs extra configs and more commands to get to the public key etc. Yubikey’s are great but block an USB port. And then there is the age old question for me: One SSH key per User for all services? One key per machine for all services? Or one key per service? This year I started to play around with the 1Password ssh-agent feature (bit warden has it as well as far as I know) If you're ok with allowing all your keys being listed in the agent this works pretty easy out of the box. I never liked the fact that the default recommended way to use ssh is to use an agent that just has multiple keys which can be tested one after another and in most cases stay unlocked there after first use for the rest of the session. I configured around to make sure that I explicitly use one key for one specific service. But that is sadly extra configuration etc etc.

I believe that in 1Passwd you can define / preselect a key per host now. So you can pinpoint key -> host. Some hosts have firewall rules that will block after X attempts were X might be low.

However the agent still has access to all your keys, obviously.

Fundamental services like DNS, which was designed as distributed going down was the cause last 2 outages and really need to think of alternatives methods to ensure resilience. Shift left, design better.

This isn't such a great idea for personal SSH or GPG keys that should be locked away in physical hardware thing that need to be moved to other devices/machines. What security processors are great for is corporate machine, system/service, and user key management IdM/MDM processes that need secret storage.

Furthermore, with portable devices like Yubikey it's possible to create a master Certify-only GPG key where the sub Signature/Encryption/Authentication-subkeys live on the Yubikey. The encrypted C private key part with the S/E/A stubs still needs to be backed-up to some durable, versioned storage that isn't tied to one device.

Finally, use GPG for SSH. And definely avoid file-based SSH local private key management for wherever possible for anything substantial because it doesn't scale well.