You'll want to set up hybrid identity, which does mean you will need to synchronize user details into Azure AD, but you can then leverage ADFS for actual authentication. You do not necessarily have to sync password hashes with Azure AD for this, and you can also scope it to a subset of users in your on-premise AD.

Alternatively, you can also configure Microsoft 365 with an internet domain you own and just have people directly sign into that - which is the alternative to a hybrid identity - a cloud only identity. You do not have to do anything then, but obviously there won't be any single sign on and all those other fancy things you could have with ADFS.