Questions about Pacemaker Clones

http://www.gossamer-threads.com/lists/linuxha/pacemaker/54575

Q: Hi All, can any one help me in understanding the difference between anonymous and globally unique clone.  anonymous Clones are supposed to behave identically everywhere they are running but globally unique clone can behave differently .

A: Right.
The best way to think about it is "Does the resource agent do anything based on the clone number?" (The clone number is the bit after the
colon in the resource's name. eg. nfs-client:1).
If yes, then your clone _must_ be of the "globally unique" type.
Common reasons for needing to answer "yes" to the question above:
- The ability to run more than one instance per node
- Using the clone number as part of a hash function to perform load balancing (eg. a cloned IP address)
Its worth mentioning that all clones of LSB resources are anonymous, because there can only be one copy active per node (specifically the
status function wont function correctly for more than one copy).
I believe drbd is another one that must be anonymous since the kernel module can only be loaded once per node.

Q: my understanding is, every resource agent has some sets of functions like start, stop ,monitor. as it will be just executing start, stop according to the resource's action . so if i do start on anonymous clone then it will be started on all nodes. but  in case of globally unique i can start , stop clone's resources on  individual node . is my understanding right?

A: No.

Neither type of resource allows you to place a specific instance on a particular node, this is by design.
The difference is that anonymous clones only allow you to have one instance per node.
Internally, this allows the PE to make certain assumptions when managing the resource.

http://www.gossamer-threads.com/lists/linuxha/pacemaker/55411?do=post_view_flat#55411

Q:

Hi All,
I am a little confused about globally-unique clones, since there can be no instance attributes for a clone how do you tell each clone that it is unique ?
My use case is that we need to run N instances of a mail server, each mail server is unique in that it serves a specific domain, two mail server can never run on the same host. In order to provide HA we need to tell the mail server instance what domain to serve (or what filesystem to mount in order to get the right data/config).
I was thinking that using globally-unique clones would be the way to manage this (it makes the mutual exclusion rule easy: clone-node-max=1) but I don't see how to make each instance unique.
... then again I could be mis-using the concept...

A:

The typical usage in this case would be to make all mail servers serve all domains (virtual domain support) and then run N instances across those N servers. Then there is no per-server unique information to deal with. Then you can run, for example, load sharing between the nodes using iptables CLUSTERIP (since all nodes would then be serving the same data) and put a constraint in the CIB that says that if the clone fails on a node, pull that node from the load-sharing config by stopping or moving away your load-sharing resource. At least, that's how I would do it.

Q:

Unfortunately this is not possible due to the underlying database the mail server uses - but thanks for the idea anyway.
So back to the original problem - am I mis-using the globally-unique clone concept ?

A:

To send in unique information, you will need to create individual instances of each. Then you will need to use location constraints to lock them down to individual nodes.
Alternatively, you could re-write the RA so that you can pass it a host-to-parameter mapping string like:
name="mydata" value="node1:www.example.com node2:www.example2.com node3:www.example3.com"
Then, inside the RA, check the hostname and use the domain that is attached to that hostname.

The idea behind unique vs. non-unique is best illustrated by example.
Take a CLUSTERIP resource...
Based on some criteria (usually the source address), it allocates all
requests into a bucket from 0..(N-1), where N ::= clone-max
So when we ask "is the resource running here", we're really asking: is
a specific bucket being served by this machine?
The easiest way to do this we set up a mapping between clone instance
(the bit after the colon) and the per-instance data.
In this case, the per-instance information is a bucket number so the
mapping is trivial.
But what it means is that myClone:0 running on nodeA is not the same
as myClone:1 on nodeB or indeed myClone:1 running on nodeA.
Each instance of the clone is considered unique and in order to
function correctly, requires globally-unique=true.
As an aside, only clones with globally-unique=true can set clone-node-max > 1.
(Perhaps the option's name could be clearer, I'm open to suggestions)
A counter example would be an LSB resource such as named.
There is only ever one copy of the service per machine and they all
serve up the same dataset, the instances are _not_ unique
(globally-unique=false) and the instance number has no meaning for the
resource.
This simplifies the resource agent since it can just look for the
named process to be running (named:0 running on nodeA is the same as
named:1 running on nodeA).
The cluster also handles anonymous clones differently, it only checks
for one copy per machine and doesn't initiate recovery if named:0 is
running on both nodeA and nodeB (instead it internally renames to use
a free "slot").
Other possible uses for clones globally-unique=true...
* Add the instance number to a base address to create an IP address range
* Map the instance number to a module (eg. in Tomcat)
* Map the instance number to a config file (eg. Apache)
* Map the instance number to a path (eg. NFS export)
Hope that helps.

> Hi All,
>
> I am a little confused about globally-unique clones, since there can be no instance attributes for a clone how do you tell each clone that it is unique ?
The globally-unique=true option.
Then, inside your resource agent, you need to set up a mapping between
the instance number and the per-instance data.
> My use case is that we need to run N instances of a mail server, each mail server is unique in that it serves a specific domain, two mail server can never run on the same host. 營n order to provide HA we need to tell the mail server instance what domain to serve (or what filesystem to mount in order to get the right data/config).
>
> I was thinking that using globally-unique clones would be the way to manage this (it makes the mutual exclusion rule easy: clone-node-max=1) but I don't see how to make each instance unique.
>
> ... then again I could be mis-using the concept...
Its a valid use-case.
As someone said below, you could set
name="mydata" value="node1:www.example.com node2:www.example2.com
node3:www.example3.com"
convert that into an array and use the clone number to index into the array.
Or set:
name=domain0 value=www.example0.com
name=domain1 value=www.example1.com
name=domainN value=www.exampleN.com
and simply look up the value of domain$CRM_meta_clone in the environment
Just make sure you include error checking so that the cluster doesn't
blow up if clone-max > number of domains.