I think as of earlier this year a feature was added in buildx to do just this.
If you have dockerfile 1.4+ and buildx 0.8+ you can do something like this:
docker buildx build --build-context othersource= ../something/something .
Then in your docker file you can use the from command to add the context
ADD –-from=othersource . /stuff
See this related post.