i using gnu make manage data processing work flow, expressing dependencies 10000+ files , 100+tb. after feature express non-file targets in makefile, such datasets in hdf5, tables in sqlite, (non-conventional) files in hdfs, etc. example:
# process dataset /src15 in pass1.h5 , save /evt15 in pass2.h5. pass2.h5(/evt15): pass1.h5(/src15) ./process.py $^ -o $@
a close feature archive members targets, limited,
this construct available in targets , prerequisites, not in recipes!
i turned gnu make extensions, either c or scheme. however, examples in manual customizing functions. 1 article states that
in future might possible make more extensive plugins, use sha1 hashes instead of timestamps, etc. moment though adding simple functions makefiles both powerful , interesting.
therefore @ least gnu make 4.0, extension mechanism limited functions.
question:
- is possible define non-file targets , recipes present extension mechanism of gnu make?
- if not, hints hacking gnu make achieve it?
- if complex, drop-in replacement of gnu make hack with? example, easier achieve makepp?
note: drake has non-file targets support, lacks pattern rules. writing out 10000+ recipes explicitly tedious, loops.
Comments
Post a Comment