Yes, this will have to be done. But the context length of most modern LLMs is huge these days so should not be a big issue.