i work spark often, , save me lot of time if compiler ensure type serializable.
perhaps type class?
def foo[t: isserializable](t: t) = { // stuff requiring t serializable }
it's not enough constrain t <: serializable. still fail @ runtime. unit tests substitute, can still forget them, when working big teams.
i think impossible @ compile time without types being sealed.
yes, possible, not in way you're hoping. type class isserializable
provide mechanism convert t
value of type is guaranteed serializable
, again,
trait isserializable[t] { def toserializable(t: t): string def fromserializable(s: string): option[t] }
but, of course, alternative type class based serialization mechanism in it's own right, making use of jvm serialization redundant.
your best course of action lobby spark support type class based serialization directly.
Comments
Post a Comment