Design an inference batching system for a single GPU that can handle up to 100 inputs per batch while users wait synchronously, maximizing utilization under compute constraints.

Focus on outlining a clear architecture for the inference batching system, detailing how you would collect and batch inputs efficiently while ensuring low latency for users. Discuss the strategy for managing synchronous requests, including how you would queue inputs and determine batch sizes dynamically based on incoming request rates. Address trade-offs between latency and throughput, and explain how you would handle points of failure, such as GPU overload or input queuing delays. Lastly, consider scalability—how would you adapt your design if the number of inputs or GPUs increased, and what measures would you implement to maintain performance under varying loads?