Multispecies coalescent process


Multispecies Coalescent Process is a stochastic process model that describes the genealogical relationships for a sample of DNA sequences taken from several species.
Gene tree is a binary graph that describes the evolutionary relationships between a sample of sequences for a non-recombining locus. Species tree describes the evolutionary relationships between a set of species, assuming tree-like evolution. However, several processes can lead to discordance between species and gene trees. The Multispecies Coalescent model provides a framework for inferring species phylogenies while accounting for ancestral polymorphism and gene tree-species tree conflict.
The process is also called the Censored Coalescent.

Multispecies Coalescent

The probability density of the gene trees under the multispecies coalescent model is discussed along with its use for parameter estimation using multi-locus sequence data.

Assumptions

The species phylogeny is assumed to be known. Complete isolation after species divergence, with no migration, hybridization, or introgression is also assumed. We assume no recombination so that all the sites within the locus share the same gene tree.

Data and Model Parameters

The model and implementation of this method can be applied to any species tree. As an example, the species tree of the great apes: human, chimpanzee, gorilla and orangutan is considered. The topology of the species tree, O)), is assumed known and fixed in the analysis. Let be the entire data set, where represent the sequence alignment at locus, with for a total of loci.
The population size of a current species is considered only if more than one individual is sampled from that species at some loci.
The parameters in the model for the example of Figure 1 include the three divergence times, and and population size parameters for humans; for chimpanzees; and, and for the three ancestral species.
The divergence times are measured by the expected number of mutations per site from the ancestral node in the species tree to the present time.
Therefore, the parameters are.

Likelihood-based inference

The gene genealogy at each locus is represented by the tree topology and the coalescent times. Given parameters, the probability distribution of is specified by the coalescent process under the model given by
The probability of data given the gene tree and coalescent times at the locus, is the Felsenstein's phylogenetic likelihood. Due to the assumption of independent evolution across the loci,
By Bayesian inference based on the joint conditional distribution
Then, the posterior distribution of is given by
where the integration represents summation over all possible gene tree topologies and integration over the coalescent times at each locus.

Distribution of gene genealogy derived from censored coalescent process

The joint distribution of is derived directly in this section. Two sequences from different species can coalesce only in one populations that are ancestral to the two species. For example, sequences H and G can coalesce in populations HCG or HCGO, but not in populations H or HC. The coalescent processes in different populations are different.
For each population, the genealogy is traced backward in time, until the end of the population at time, and the number of lineages entering the population and the number of lineages leaving it are recorded. For example, and, for population H. This process is called a censored coalescent process because the coalescent process for one population may be terminated before all lineages that entered the population have coalesced. If the population consists of disconnected subtrees or lineages.
With one time unit defined as the time taken to accumulate one mutation per site, any two lineages coalesce at the rate. The waiting time until the next coalescent event, which reduces the number of lineages from to has exponential density
If, the probability that no coalescent event occurs between the last one and the end of the population at time ; i.e. during the time interval. This probability is and is 1 if.
In addition, to derive the probability of a particular gene tree topology in the population, if a coalescent event occurs in a sample of lineages, the probability that a particular pair of lineages coalesce is.
Multiplying these probabilities together, the joint probability distribution of the gene tree topology in the population and its coalescent times as
The probability of the gene tree and coalescent times for the locus is the product of such probabilities across all the populations. Therefore, the gene genealogy of Figure 1, we have