Support OpenSSL 3.0

OpenSSL 3.0 is scheduled to be released later this year. It is a major version bump from 1.1 and contains architecture changes that affect Ruby/OpenSSL.

From https://www.openssl.org/policies/releasestrat.html:

The following alpha and beta releases for OpenSSL 3.0 are currently scheduled. Note that these dates are subject to change and alpha or beta releases may be inserted or removed as required:
alpha1, 2020-03-31: Basic functionality plus basic FIPS module
alpha2, 2020-04-21: Complete external provider support (serialization, support for new algs, support for providers which only include operations in a class)
alpha3, 2020-05-21: Aiming to test the API completeness before beta1 freezes it)
beta1, 2020-06-02: Code complete (API stable, feature freeze)
betaN: Other beta releases TBD
Final: 2020 early Q4

The design is outlined in the web page:

https://www.openssl.org/docs/OpenSSL300Design.html

Unlike OpenSSL 1.0 -> 1.1, not so many changes are required to make it just compile, but a lot of deprecation warnings are generated while compiling and many test cases are currently failing when compiled against OpenSSL's master.

OpenSSL::HMAC needs a rewrite with the EVP API as it currently uses the low-level HMAC_*() functions.
Subclasses of OpenSSL::PKey, such as RSA or DSA, provide access to those low-level functions. The following methods need rewrite:
- PKey::*#generate (and an overload of .new)
  - This can probably be implemented in pure-Ruby with PKey.generate_parameters and PKey.generate_key.
- Low-level sign/verify methods that take prehashed values.
  - RSA#{private,public}_{encrypt,decrypt}
  - DSA#syssign and #sysverify
  - EC#dsa_sign_asn1 and #dsa_verify_asn1
The ENGINE API is deprecated in favor of "Provider"s.

Updated TODO list as of alpha14:

OpenSSL::HMAC, Use the EVP API
- No incompatibilities expected for Ruby programs.
- Done by hmac: migrate from the low-level HMAC API to the EVP API #371.
OpenSSL::PKey, Parameters/key generation with the EVP API
- No incompatibilities expected.
- Done by pkey: use high level EVP interface to generate parameters and keys #397.
  - Added OpenSSL::PKey.generate_key and OpenSSL::PKey.generate_parameters.
  - Rewritten OpenSSL::PKey::{RSA,DSA,DH}.{new,generate}.
OpenSSL::PKey::*, Low-level methods for signature/encryption/key agreement
- RSA#{private,public}_{encrypt,decrypt}, DSA#syssign, DSA#sysverify, EC#dsa_sign_asn1, and EC#dsa_verify_asn1.
- No incompatibilities expected.
- Done by pkey: implement PKey#encrypt, #decrypt, #sign_raw, #verify_raw, and #verify_recover #382.
  - Added OpenSSL::PKey.{encrypt,decrypt,sign_raw,verify_raw,verify_recover}.
OpenSSL::PKey::*, Getters for parameters/key components
- RSA#{n,e,d,p,...}, etc.
- No incompatibilities expected.
- Needs a rewrite with EVP_PKEY_get_params() family (only exists in 3.0)
OpenSSL::PKey::*, Setters for parameters/key components
- {RSA,DSA,DH}#set_* and EC#{private_key=,public_key=,group=}
- Feature removed without replacement. Keys are now immutable once created - all components must be specified at once.
- Use cases definitely exist, a new interface is required.
  - EVP_PKEY_fromdata() requires the caller to specify what the pkey is: parameters only, public key only, or private key?
  - OpenSSL::PKey.new_private_key("RSA", n: 123, e: 456, d: 789)?
OpenSSL::PKey::{RSA,DSA}, Private/public key decoders for unpopular/non-standard formats
- OpenSSL::PKey::DSA.new accepts PEM encoded "DSAPublicKey" format ("DSA PUBLIC KEY" header).
- OpenSSL::PKey::RSA.new accepts DER/PEM encoded "RSAPublicKey" format ("RSA PUBLIC KEY" header).
- ruby-openssl has no ability to export keys in such formats as of [Bug #4422].
- Would it be OK to drop them? We need to write our decoder using OpenSSL::ASN1.decode if we want to keep it.
OpenSSL::PKey::*, Methods that need a rewrite, which can be done with existing (<= 1.1.1) API
- #to_text
- #public_key (the method to create a copy with private components removed)
- DH#params_ok? and EC#check_key
- Done in Use EVP API in more places #436.
OpenSSL::PKey::*, Methods that need a rewrite, which can be done in a compatible way, but requires new functions in 3.0.0
- #params
- #initialize_copy - annoying because EVP_PKEY_dup() only exists in OpenSSL 3.0.
- #private? and #public?
OpenSSL::PKey::{DH,EC}, #generate_key!
- Feature removed without replacement. Keys are now immutable once created.
- Print deprecation warning and suggest using OpenSSL::PKey.generate_key(pkey_obj_without_key_components).
OpenSSL::PKey::EC::Group and OpenSSL::PKey::EC::Point
- Effectively removed without replacement, though most of the functions are not explicitly marked as deprecated.
- Because the low-level key generation is deprecated and the EVP API can't create keys using an EC_GROUP object, there is no use for this class.
- Information that used to be obtained through EC_GROUP object (curve's name or parameters) are directly obtained from EVP_PKEY object with EVP_PKEY_get_params().
OpenSSL::Engine
- The entire ENGINE API is deprecated in favor of Providers, but it's new and is a completely different concept.
- ENGINE should work if we ignore the deprecation warnings, but I doubt if there will be any ENGINE implementations that work with OpenSSL 3.0 anyway.
OpenSSL::SSL, tmp_dh_callback
- The callback used on the server side for ephemeral DH. Called during the handshake, but there is no reason for it to be a callback (anymore).
- Replaced by SSL_CTX_set_tmp_dh() or SSL_CTX_set_dh_auto().
- Added SSLContext#tmp_dh= by ssl: add SSLContext#tmp_dh= #459.

@rhenium

This is awesome, thanks for your hard work @rhenium!

Use `OpenSSL 1.1` until `OpenSSL for Ruby` will support `OpenSSL 3.0`

Use openssl 1.1.1m (Ruby 2.7.2 official uses 2.1.2.. but conan only h…

Revert "feat: Build packages on Ubuntu 22.04 (Jammy)"

@rhenium

@rhenium

I just updated two items:

First, the below test is skipped for OpenSSL 3. I removed the 'pend', and it passes with the three Ruby builds using OpenSSL 3, Windows mswin, Ubuntu 22.04 3.1 & head.

openssl/test/openssl/test_hmac.rb

Lines 23 to 29 in 8752d9e

    
           def test_dup 
        
             pend "HMAC#initialize_copy is currently broken on OpenSSL 3.0.0" if openssl?(3, 0, 0) 
        
             h1 = OpenSSL::HMAC.new("KEY", "MD5") 
        
             h1.update("DATA") 
        
             h = h1.dup 
        
             assert_equal(h1.digest, h.digest, "dup digest") 
        
           end

Secondly, I updated the Actions test-openssls jobs, 1.1.1l to 1.1.1q, and 3.0.1 to 3.0.5. They also passed. JFYI, 3.0.1 testing froze before I updated these.

Not sure if you want PR's, three lines of code, also added timeouts to the test steps.

Thanks for your work on this. A lot of non-trivial work...

First, the below test is skipped for OpenSSL 3.

Yes, we can remove it now. The problem was apparently fixed by OpenSSL 3.0.2 (openssl/openssl@9a4e7d8).

Secondly, I updated the Actions test-openssls jobs, 1.1.1l to 1.1.1q, and 3.0.1 to 3.0.5. They also passed. JFYI, 3.0.1 testing froze before I updated these.

Not sure if you want PR's, three lines of code, also added timeouts to the test steps.

Yes, please!

But that's a good point, because

require 'openssl'

ec_key = OpenSSL::PKey::EC.generate('prime256v1')
cipher = OpenSSL::Cipher.new('aes-256-gcm')
pem = ec_key.export(cipher, 'key_password')

p OpenSSL::PKey.read(pem, 'key_password')

fails with

/tmp/3.rb:7:in `read': Could not parse PKey: bad decrypt (OpenSSL::PKey::PKeyError)
	from /tmp/3.rb:7:in `<main>'

Maybe it's covered by unchecked points above, but dunno.

EDIT: okay, gcm is hard and I can't get it working even with openssl-1.

Do you have any suggestions to update the Ruby web-push library to use OpenSSL v3?

I can replace OpenSSL::PKey::EC.new with OpenSSL::PKey::EC.generate, but then I find it hard to update this file to v3:

https://github.com/pushpad/web-push/blob/master/lib/web_push/vapid_key.rb

The gem has good test coverage and is already updated to use the openssl gem v3, but tests pass only on with the C library v1.1 (and not with C library v3).

Any suggestions?

Don't use OpenSSL-3, `openssl` gem doesn't support it yet

# input: the base64 string of the public key and the base64 string of private key
# output: an OpenSSL::PKey::EC with that keys

public_key = OpenSSL::PKey::EC::Point.new(group, to_big_num(public_key_base64))
private_key = to_big_num(private_key_base64)

# ... then how do you generate the OpenSSL::PKey::EC from that (in OpenSSL 3)?

This is the only workaround that we have found:
https://github.com/pushpad/web-push/pull/2/files#diff-e8e36e8a5282b2fbf5503e699e222f3d5413c86cdbb94ecdd13148ea0c59e5f5R90

Which is quite complex compared to the straightforward assignment that you could do with OpenSSL 1.1.

Is there a simpler alternative to create an OpenSSL::PKey::EC from existing keys (in base64) in OpenSSL 3?

OpenSSL::PKey::, Setters for parameters/key components
{RSA,DSA,DH}#set_ and EC#{private_key=,public_key=,group=}
Feature removed without replacement. Keys are now immutable once created - all components must be specified at once.

Would it make sense to replace these methods with a new method like set_keys or new_from_keys that assigns both keys at the same time (since they are immutable)? Would you consider adding it to Ruby? Or what is the new recommended way for assigning the existing keys?

Would it make sense to replace these methods with a new method like set_keys or new_from_keys that assigns both keys at the same time (since they are immutable)?

#555 is working on it, which provides access to EVP_PKEY_fromdata() added in OpenSSL 3.0. It currently (as of openssl gem v3.1) has to be done by making the ASN.1 encoding as in this PR:

This is the only workaround that we have found:
https://github.com/pushpad/web-push/pull/2/files#diff-e8e36e8a5282b2fbf5503e699e222f3d5413c86cdbb94ecdd13148ea0c59e5f5R90

As this is related to supporting OpenSSL 3, I just put the issue tickets and PR to support FIPS mode on OpenSSL 3

FYI OpenSSL 1.1.1 (the last version before 3.0) is set to be EOL in less than 2 months.

I first want to say I appreciate all the work that has gone into adding 3.0 support so far (as well as all the other work on this gem).

From my limited perspective, it looks like full support is unlikely to be completed before the EOL deadline. Is this a reasonable assumption? If so, it seems like something many users will want to know ahead of time.

Although the occasional OpenSSL-3.x issue is still spotted and fixed, according to https://github.com/ruby/openssl/blob/master/History.md OpenSSL 3.0 has been supported since v3.0.0 in Dec 24, 2021.

Meanwhile projects that consume this library are looking at this issue still being open as a reason they can't support OpenSSL 3.x yet. Can this issue be marked closed?

We also got confused by this open issue and the incomplete 3.0 milestone: https://github.com/ruby/openssl/milestone/2
We didn't move to OpenSSL 3 because we got the impression it wasn't ready for production yet.

@rhenium

Agreed, @rhenium are you OK to close this issue and the milestone?
And if there any remaining TODO for OpenSSL 3.0 then I would suggest to file a new issue with just that.

hmac: use EVP_PKEY_new_raw_private_key() if available

pkey/ec: check existence of public key component before exporting

Navigation Menu

Repository navigation

Support OpenSSL 3.0 #369

26 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	def test_dup
	pend "HMAC#initialize_copy is currently broken on OpenSSL 3.0.0" if openssl?(3, 0, 0)
	h1 = OpenSSL::HMAC.new("KEY", "MD5")
	h1.update("DATA")
	h = h1.dup
	assert_equal(h1.digest, h.digest, "dup digest")
	end

Support OpenSSL 3.0 #369

Description

Activity

rhenium commented on Apr 13, 2021

zzak commented on Apr 13, 2021

pvalena commented on May 24, 2021

MSP-Greg commented on Jul 26, 2022

rhenium commented on Jul 27, 2022

26 remaining items

ojab commented on Nov 2, 2022

collimarco commented on Dec 8, 2022

collimarco commented on Dec 26, 2022

collimarco commented on Dec 27, 2022

rhenium commented on Dec 28, 2022

junaruga commented on Mar 23, 2023

mlarraz commented on Jul 18, 2023

hlein commented on Dec 10, 2023

thomthom commented on Dec 11, 2023

eregon commented on Dec 11, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions