I’ve set up a batch consumer of an event ExampleEvent and set it to have ConcurrencyLimit = 16, MessageLimit = 100, TimeLimit = 5 sec and grouping by ExampleEvent.GroupingKey field.
This works fine and shows up on Rabbit as a consumer with PrefetchCount = 1600. The thing is that I have a multi pod environment and have more than one consumer like that connected to the same queue. I read that Rabbit distributes messages using a round robin algorithm, however I also read that it will try to saturate one consumer until the PrefetchCount limit of unacked messages is reached. This seems to not be the case (or maybe I’m missing something?).
Now the main problem is that some messages that could’ve been batched with other messages on Consumer 1 are being sent to Consumer 2 (replica). According to logs, they were taken within less than a second of eachother and the message limit was far from being reached.
Is it possible to somehow force them to be batched on Consumer 1?
>Solution :
When running multiple instances/pods of the same service, messages on the queue will be load balanced across all instances/pods. There is no way to control the direction of those messages in RabbitMQ so that they only go to a specific instance.