Design an end-to-end batching system for LLM queries.

Focus on outlining the architecture of the batching system, including how you would handle incoming requests and manage the queue. Discuss the API design, specifying endpoints for submitting queries and retrieving responses, and consider how to optimize for latency and throughput. Address potential challenges such as handling variable query sizes and prioritizing requests. Finally, think about how to scale the system and ensure reliability, including error handling and fallback mechanisms.