Fails to sign with 4096 bit RSA #80
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
I am using a Xiao RP2350 with PicoHSM FW 5.4 as provided in the release notes here.
I am executing the following commands on Windows using OpenSC 0.26.1:
pkcs11-tool -l --pin <mypin> --keypairgen --key-type rsa:4096 --id 1 --label 001openssl req -x509 -new -nodes -engine pkcs11 -key 31 -keyform engine -out 001.crt -days 9125 -config openssl.cnf -subj "%SUBJ%" -reqexts ca_extThis, however, fails after sending the APDU for signing:
In Wireshark I can see two things I would expect different (without checking the actual specification):
First: The sequence number stays the same. Is that correct?
Second: There is some unknown message type from Pico HSM to PC. What means 0xBD? I can see a lot of 0x55 in other parts, but that does not seems to be an issue.
Can you try with a shorter key like 1024?
1024: OK
2048: OK
3072: FAIL
4096: FAIL
Is there anything else I can do to assist you with the bug fix?
I’ll debug myself. For curiosity, are you generating the 4k key in the device or it’s imported? How nuch time does it take?
I got the 4096 bit key generated from within the device. I am not sure how long that takes. It starts and fails similar to the signing process. However, the result is not important at the USB interface so I can check after a while for the correctly generated key. I suspect that the keep-alive USB function has some issue. I.e. any USB operation taking longer than 1.5s? fails. Fortunately, the key generation does continue in the background :)
Try this snippet and report the result:
test.zip
Is the key 33 a rsa 4096?
No, 1024 because 4096 fails.
Btw, I am not sure why but keypairgen id 3 is turned into id 33.
So you tried the snippet with a rsa 4096 and failed, right? In which step? Which error?
If you ask me that way... key generation fails for every RSA key size:
Same each (example above for 1024 bit):
pkcs11-tool -l --pin 0815 --keypairgen --key-type rsa:1024 --id 3 --label testpkcs11-tool -l --pin 0815 --keypairgen --key-type rsa:2048 --id 3 --label testpkcs11-tool -l --pin 0815 --keypairgen --key-type rsa:3072 --id 3 --label testpkcs11-tool -l --pin 0815 --keypairgen --key-type rsa:4096 --id 3 --label testHowever, it completes in the background after a while. But usually, the device needs to be plugged again after that to work correctly.
Just for reference:
If I do all the steps here with the 4096 bit RSA key it fails at:
pkcs11-tool --id 31 --sign --pin 0815 --mechanism SHA1-RSA-PKCS -i data -o data.sigwith the following error:
Previous test have been done with Windows 7 x64.
Running the same with Windows 11 24H2 x64 gives no errors.
In particular the following issues do not occur:
Both tests have been done with the same OpenSC build.
I am not sure whether this is an OpenSC or a PicoHSM issue...
Really strange. Also in macOS and Linux works fine too. I don’t know what changed between windows builds.
P.S.: RSA 4096 creation took around 300s on that device.
I suggest closing this issue with a remark in the readme for the minimal supported/tested Windows version.
Or do you plan to fix it?
To me it’s strange that in windows doesn’t work 4k, specially since 1k or 2k work well and all them work in macos/linux too. There’s nothing special different between 2k and 4k.
As mentioned before, Windows 7 seems to handle a couple of things differently.
Neither did the timeout function work properly nor the labeling.
Nevertheless, the question remains whether this project wants to support Windows 7 or not.
Perhaps related with #95 ?
Hard to say. Sounds a bit different.
Not the problem but the cause, an underlaying issue with returned content.