Building with --separate-weights is making too many assumptions

[aknoerig]

I was playing around with cog build on a custom stable-diffusion-v1-5 model and got bitten by adding the --separate-weights flag. There is hardly any documentation on how it is achieving the image layer separation, and so I only found out the hard way. 😄

Turns out it silently extends .dockerignore apparently by auto-guessing the model checkpoint paths (plus some default paths). As the checkpoint file structure is quite deep and complex in this case, this broke the build process, including any following one without the flag:

The following section was added to my .dockerignore (actually multiple times, for every build using --separate-weights).

# generated by replicate/cog
__pycache__
*.pyc
*.pyo
*.pyd
.Python
env
pip-log.txt
pip-delete-this-directory.txt
.tox
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.log
.git
.mypy_cache
.pytest_cache
.hypothesis
checkpoint/safety_checker
checkpoint/safety_checker/**/*
checkpoint/text_encoder
checkpoint/text_encoder/**/*
checkpoint/unet
checkpoint/unet/**/*
checkpoint/vae
checkpoint/vae/**/*

While the models in this case do reside inside the checkpoint folder, this pattern only captures a subset of the files, so that the build eventually fails.

My suggestion would be to make this configuration explicit, for example by letting the user specify a path to the model root. And of course to add some documentation about what is actually going on.