Link datafusion as dynamic library

[sergiimk]

Related to: apache/datafusion#6470

Description

• Created a tiny fork of datafusion that uses a technique from bevy engine to add dynamic linking option as a crate feature
• The datafusion-dynamic is a default feature in kamu-cli and through workspace feature unification it enables dynamic linking in all tests
• Release build disables default features to link statically as before
• dev build size impact:
  • reduces from 37 to 25 GiB without debug symbols
  • reduces from 112 GiB to 56 GiB with full debug symbols

How it works

In the fork:

• datafusion becomes a facade library that links:
  • datafusion-core - the main guts
  • (optionally via feature) datafusion-dylib
• datafusion-dylib is a dynamic crate that also links the entire datafusion-core
• When the feature is enabled and datafusion-dylib is linked - the linker sees two sets of same symbols (one from a static and one from dynamic lib) and it automatically prefers the dynamic lib

Downsides

• Fork maintenance - should be simple, the worst part is needing to propagate all features from the facade into guts crate
• Cargo patching - currently needs to list ALL datafusion libraries

Ideally we would some day upstream this into datafusion and the hacky stuff will go away.