Stuck when generating a RSA key #115

Closed
opened 2025-12-08 01:28:14 +08:00 by michaelrrrrrr · 5 comments

I am using a ESP32-S3 board to flash the Pico HSM firmware. The OS is Windows 11 Pro.

I followed the steps in the doc/usage.md. The initialization is good. The init cmd is as below:
python pico-hsm-tool.py initialize --pin 275809 --so-pin E11DEBA1706B3903

And then I modify the VID/PID using Pico Commissioner web tool. This step is also good. (I saw the success sign.)

When I started generating RSA key following the instruction in usage.md, it went wrong. This process took over 20 min.

The keypair gen cmd is pkcs11-tool -l --pin 275809 --keypairgen --key-type rsa:1024 --id 1 --label "k1" . To speed up the process, I used the rsa:1024, but it didn't work.

I had no idea what happened, so I tried the keypair generation process several times. If I unplug the USB key within a couple of minutes, the cmd.exe prompts like this:
C:\Users\michaelren\Repositories\pico-hsm\tools>pkcs11-tool -l --pin 275809 --keypairgen --key-type rsa:1024 --id 1 --label "k1"
Using slot 0 with a present token (0x0)
error: PKCS11 function C_GenerateKeyPair failed: rv = CKR_DEVICE_REMOVED (0x32)
Aborting.

However, once I waited too long to know how much time the generation process took, and finally I removed the USB, this time the output info changed to:
C:\Users\michaelren\Repositories\pico-hsm\tools>pkcs11-tool -l --pin 275809 --keypairgen --key-type rsa:1024 --id 1 --label "k1"
Using slot 0 with a present token (0x0)
error: PKCS11 function C_GenerateKeyPair failed: rv = CKR_GENERAL_ERROR (0x5)
Aborting.

P.S. Tried different ESP32-S3 boards, the results were same.

I am using a ESP32-S3 board to flash the Pico HSM firmware. The OS is Windows 11 Pro. I followed the steps in the doc/usage.md. The initialization is good. The init cmd is as below: `python pico-hsm-tool.py initialize --pin 275809 --so-pin E11DEBA1706B3903` And then I modify the VID/PID using Pico Commissioner web tool. This step is also good. (I saw the success sign.) When I started generating RSA key following the instruction in [usage.md](https://github.com/polhenarejos/pico-hsm/blob/master/doc/usage.md#keypair-generation), it went wrong. This process took over 20 min. The keypair gen cmd is `pkcs11-tool -l --pin 275809 --keypairgen --key-type rsa:1024 --id 1 --label "k1"` . To speed up the process, I used the rsa:1024, but it didn't work. I had no idea what happened, so I tried the keypair generation process several times. If I unplug the USB key within a couple of minutes, the cmd.exe prompts like this: C:\Users\michaelren\Repositories\pico-hsm\tools>pkcs11-tool -l --pin 275809 --keypairgen --key-type rsa:1024 --id 1 --label "k1" Using slot 0 with a present token (0x0) error: PKCS11 function C_GenerateKeyPair failed: rv = CKR_DEVICE_REMOVED (0x32) Aborting. However, once I waited too long to know how much time the generation process took, and finally I removed the USB, this time the output info changed to: C:\Users\michaelren\Repositories\pico-hsm\tools>pkcs11-tool -l --pin 275809 --keypairgen --key-type rsa:1024 --id 1 --label "k1" Using slot 0 with a present token (0x0) error: PKCS11 function C_GenerateKeyPair failed: rv = CKR_GENERAL_ERROR (0x5) Aborting. P.S. Tried different ESP32-S3 boards, the results were same.

Yes, ESP32 is really painful doing RSA. Use it for ECC instead or generate it offline and import.

Yes, ESP32 is really painful doing RSA. Use it for ECC instead or generate it offline and import.

Yes, ESP32 is really painful doing RSA. Use it for ECC instead or generate it offline and import.

So the ESP32 is totally out for RSA, right?

I guess RP2350 is better option for Pico HSM.

> Yes, ESP32 is really painful doing RSA. Use it for ECC instead or generate it offline and import. So the ESP32 is totally out for RSA, right? I guess RP2350 is better option for Pico HSM.

load("keymanager/keymanager.js");

SmartCard-HSM Version 6.0 on JCOP Free memory 1048576 byte
Issuer Certificate : CVC id-AT DV (official domestic) CAR=ESPICOHSMCA00002 CHR=ESPICOHSMDV00002 CED=April 6, 2025 CXD=April 6, 2026
Device Certificate : CVC id-AT Terminal CAR=ESPICOHSMDV00002 CHR=ESPICOHSMTR692WX CED=December 7, 2025 CXD=December 7, 2026
Default Key Domain : 57C6E455BB768A1E76FFF38BD036A0085798613918731A4D5150EAA36C4E0C8E
Creating outline...
org.mozilla.javascript.EcmaError: ReferenceError: "OutlineNode" is not defined. (D:\Program Files\scsh_with_JRE\keymanager\keymanager.js#1653)
at D:\Program Files\scsh_with_JRE\keymanager\keymanager.js#1653
at D:\Program Files\scsh_with_JRE\keymanager\keymanager.js#304
at D:\Program Files\scsh_with_JRE\keymanager\keymanager.js#41
at D:\Program Files\scsh_with_JRE\keymanager\keymanager.js#3348

@polhenarejos I got this when I tried to import RSA key. I double checked the code and am sure everything goes as scs3.md. Is scsh with JRE not the compatible scsh for this?

>load("keymanager/keymanager.js"); SmartCard-HSM Version 6.0 on JCOP Free memory 1048576 byte Issuer Certificate : CVC id-AT DV (official domestic) CAR=ESPICOHSMCA00002 CHR=ESPICOHSMDV00002 CED=April 6, 2025 CXD=April 6, 2026 Device Certificate : CVC id-AT Terminal CAR=ESPICOHSMDV00002 CHR=ESPICOHSMTR692WX CED=December 7, 2025 CXD=December 7, 2026 Default Key Domain : 57C6E455BB768A1E76FFF38BD036A0085798613918731A4D5150EAA36C4E0C8E Creating outline... org.mozilla.javascript.EcmaError: ReferenceError: "OutlineNode" is not defined. (D:\Program Files\scsh_with_JRE\keymanager\keymanager.js#1653) at D:\Program Files\scsh_with_JRE\keymanager\keymanager.js#1653 at D:\Program Files\scsh_with_JRE\keymanager\keymanager.js#304 at D:\Program Files\scsh_with_JRE\keymanager\keymanager.js#41 at D:\Program Files\scsh_with_JRE\keymanager\keymanager.js#3348 > @polhenarejos I got this when I tried to import RSA key. I double checked the code and am sure everything goes as [scs3.md](https://github.com/polhenarejos/pico-hsm/blob/master/doc/scs3.md). Is [scsh with JRE](https://www.openscdp.org/scsh3/download.html) not the compatible scsh for this?

@polhenarejos Btw, I tried the "Generate certificate and sign it" section and ran openssl installed by winget. I know this openssl is not connected with pkcs11-tool that connects with Pico HSM key, so how can I fix that?

Thx for your help.

@polhenarejos Btw, I tried the "Generate certificate and sign it" section and ran `openssl` installed by `winget`. I know this `openssl` is not connected with pkcs11-tool that connects with Pico HSM key, so how can I fix that? Thx for your help.

@polhenarejos Btw, I tried the "Generate certificate and sign it" section and ran openssl installed by winget. I know this openssl is not connected with pkcs11-tool that connects with Pico HSM key, so how can I fix that?

Thx for your help.

In case someone has same issue, I post what I've done to fix the issue I had. Hopefully, this post will help someone.

When I installed openssl using winget, I tried openssl req -engine pkcs11 -new -key 0:1 -keyform engine -out cert.pem -text -x509 -days 365 to sign the certificate. But it output as below:

PKCS11_get_private_key returned NULL
Could not find private key from org.openssl.engine:pkcs11:0:1
18150000:error:41000401:libp11:ERR_P11_error:Unable to load PKCS#11 module:p11_load.c:101:
18150000:error:13000080:engine routines:ENGINE_load_private_key:failed loading private key:crypto\engine\eng_pkey.c:79:

To fix this, download pre-compiled pkcs11.dll from OpenSC libq11 release page. For OpenSSL 3.x use the latest libp11 version. Unzip the downloaded archive, copy pkcs11.dll to somewhere under C drive. I put it in the subdir of openssl.

Then, set the environment variable OPENSSL_CNF set OPENSSL_CNF=C:\Program Files\OpenSSL-Win64\bin\cnf\openssl.cnf. Note that no need to add quotes even if the path has a space char. I also set PKCS11_MODULE_PATH to C:\Program Files\OpenSC Project\pkcs11\opensc-pkcs11.dll.

Next, edit the file openssl.cnf that is located at C:\Program Files\OpenSSL-Win64\bin\cnf (C:\Program Files\OpenSSL-Win64 is where I installed openssl). Make sure openssl_conf = openssl_init is in the file, and then add following lines at the end of the file:

[openssl_init]
engines = engine_section

[engine_section]
pkcs11 = pkcs11_section

[pkcs11_section]
engine_id = pkcs11
dynamic_path = "/path/to/openssl/lib/engines/pkcs11.dll"
MODULE_PATH = "/path/to/OpenSC/pkcs11/opensc-pkcs11.dll"
init = 0
# optionally: PIN = 123456

Next, run openssl engine dynamic -pre ID:pkcs11 -pre SO_PATH:/path/to/openssl/lib/engines/pkcs11.dll -pre LIST_ADD:1 -pre LOAD -pre MODULE_PATH:/path/to/OpenSC/pkcs11/opensc-pkcs11.dll to load the engine and point it to module. Some -pre options can be taken down.

Now, run openssl engine pkcs11 -t to check if the engine is set right. If you see:

(pkcs11) pkcs11 engine
     [ available ]

the engine setting should be passed.

> [@polhenarejos](https://github.com/polhenarejos) Btw, I tried the "Generate certificate and sign it" section and ran `openssl` installed by `winget`. I know this `openssl` is not connected with pkcs11-tool that connects with Pico HSM key, so how can I fix that? > > Thx for your help. In case someone has same issue, I post what I've done to fix the issue I had. Hopefully, this post will help someone. When I installed openssl using `winget`, I tried `openssl req -engine pkcs11 -new -key 0:1 -keyform engine -out cert.pem -text -x509 -days 365` to sign the certificate. But it output as below: ``` PKCS11_get_private_key returned NULL Could not find private key from org.openssl.engine:pkcs11:0:1 18150000:error:41000401:libp11:ERR_P11_error:Unable to load PKCS#11 module:p11_load.c:101: 18150000:error:13000080:engine routines:ENGINE_load_private_key:failed loading private key:crypto\engine\eng_pkey.c:79: ``` To fix this, download pre-compiled pkcs11.dll from [OpenSC libq11 release page](https://github.com/OpenSC/libp11/releases). For OpenSSL 3.x use the latest libp11 version. Unzip the downloaded archive, copy pkcs11.dll to somewhere under C drive. I put it in the subdir of openssl. Then, set the environment variable OPENSSL_CNF `set OPENSSL_CNF=C:\Program Files\OpenSSL-Win64\bin\cnf\openssl.cnf`. Note that no need to add quotes even if the path has a space char. I also set PKCS11_MODULE_PATH to `C:\Program Files\OpenSC Project\pkcs11\opensc-pkcs11.dll`. Next, edit the file openssl.cnf that is located at `C:\Program Files\OpenSSL-Win64\bin\cnf` (`C:\Program Files\OpenSSL-Win64` is where I installed openssl). Make sure `openssl_conf = openssl_init` is in the file, and then add following lines at the end of the file: ``` [openssl_init] engines = engine_section [engine_section] pkcs11 = pkcs11_section [pkcs11_section] engine_id = pkcs11 dynamic_path = "/path/to/openssl/lib/engines/pkcs11.dll" MODULE_PATH = "/path/to/OpenSC/pkcs11/opensc-pkcs11.dll" init = 0 # optionally: PIN = 123456 ``` Next, run `openssl engine dynamic -pre ID:pkcs11 -pre SO_PATH:/path/to/openssl/lib/engines/pkcs11.dll -pre LIST_ADD:1 -pre LOAD -pre MODULE_PATH:/path/to/OpenSC/pkcs11/opensc-pkcs11.dll` to load the engine and point it to module. Some `-pre` options can be taken down. Now, run `openssl engine pkcs11 -t` to check if the engine is set right. If you see: ``` (pkcs11) pkcs11 engine [ available ] ``` the engine setting should be passed.
Sign in to join this conversation.