If the percentage of messages that go to any given consumer is low, what
you have is probably best. As that percentage goes up (i.e. as each
message goes to more and more consumers), you'll hit a point where you're
better off with a single topic to which the producer sends only a single
message with all the intended recipients listed in a custom header
property. Each consumer would get each message and look at the header
property to determine whether it's supposed to process the message. (Maybe
it's possible to use selectors to do that filtering, but I'm not sure
there's a way to do "contains" in a selector. Or maybe you set headers
like "u1=true" for only the users that are supposed to get the message, and
then have the selector test for that header being set. But your consumer
can always do that logic itself if a selector isn't an option.)
The trade-off is between having the producer create N equivalent messages
(which increases the delay for the producer and the bandwidth it uses, plus
results in a more complicated and hard-to-monitor broker because of all the
destinations but allows consumers to download only the messages that apply
to them) versus having the producer create only a single message with N
recipients in the header to a single topic (which is obviously faster for
the producer, but requires each consumer to download the message to see if
it applies to them - and the message is larger because there are N
recipients listed in the header). As N gets closer to the total number of
consumers, the waste due to downloading messages not meant for you drops,
and even though each message is larger, it's probably still less total
bandwidth than if the producer sent N copies of the message (and it's not
"paused" sending 100,000 copies of each message). So it'll depend heavily
on whether N is large or small.
One other thing: if N is small to medium and you find a way to make
selectors work, filtering will happen on the broker and you can avoid all
the wasted bandwidth of downloading messages that aren't for each
consumer. The downside is that every one of those selectors (100,000 or
however many consumers you have) has to be evaluated against every message,
which can be a real burden on the broker. (Art warns people strongly about
the performance dangers of heavily using selectors.) So it's the ideal
solution for your producers and consumers, but it's the worst option for
the broker and before you commit to using it (especially with the number of
consumers you've described), you'd better do some thorough performance
testing to make sure the broker can handle it.
On Apr 8, 2015 2:10 AM, "William" wrote:
Thanks for reply, but our customer demand us to use mqtt protocol. So we
choose activeMQ which can receive mqtt msgs and map to jms msgs.
On Apr 8, 2015, at 3:45 PM, James Carman wrote:
Maybe use something like XMPP instead?
On Wednesday, April 8, 2015, William wrote:
Is there any best practise to sending message to a large users
(more than 100k users) ?
for example, sometime system will send “msgA” to user1, u2, u3 …
in next stage, system will send “msgB” to user2, u4, u6,
due to the requirement, it’s hard to spilt it into different topic.
My solution is creating topic for each user, and loop to send
message to them. But the effective is so low.
So I wonder if having a good idea about this requirement? thanks a lot!