A Family-Based Graphical Approach for Testing Hierarchically Ordered Families of Hypotheses
In applications of clinical trials, tested hypotheses are often grouped as multiple hierarchically ordered families. To test such structured hypotheses, various gatekeeping strategies have been developed in the literature, such as series gatekeeping, parallel gatekeeping, tree-structured gatekeeping strategies, etc..
Key words : hierarchically ordered families; gatekeeping
However, these gatekeeping strategies are often either non-intuitive or less flexible when addressing increasingly complex logical relationships among families of hypotheses.
我们想解决什么问题?
图形化拥有层次逻辑结构的假设检验族,或者说糅合了 gatekeeping策略和graphicial approach的图示法(Bretz et al. (2009) 和 Dmitrienko et al. (2008) )。相比于Burman et al. (2008)的方法,图示法是“alpha-exhaustive”的; 对于Bretz et al. (2009)提出的方法,核心是"Hypothesis-based graphical approach",此处的方法核心为"Family-based graphical approach".
In order to overcome the issue, Qiu et al. (2019) developed a new family-based graphical approach which is based on the approach introduced by Bretz et al (2009), which can easily derive and visualize different gatekeeping strategies.
In the proposed approach, a directed and weighted graph is used to represent the generated gatekeeping strategy where each node corresponds to a family of hypotheses and two simple updating rules are used for updating the critical value of each family and the transition coefficient between any two families.
Theoretically, they have shown that the proposed graphical approach strongly controls the overall family-wise error rate at a pre-specified level.
Methodology of family-based graphical approach
While testing multiple families of hypotheses, hierarchically logical restrictions among the families are often one important aspect. Thus, it is natural for us to focus more on the logical relationships at family level rather than at hypothesis level, to develop a graphical approach for visualizing conventional gatekeeping strategies for testing multiple ordered families of hypotheses.
By using the similar idea as in Kordzakhia and Dmitrienko (2013), a vertex could be used to represent a family of hypotheses instead of an individual hypothesis and a directed edge with a pre-specified weight associated with it to represent the transition relationship between two families. This approach is termed as family-based graphical approach.

Figure - A intuitive example of Hypothesis-based graphical visualization of gatekeeping procedure with truncated Holm procedure with truncation parameter
Basic notation
Suppose there are
Each family
These families
Each of the true null p-value is assumed to be stochastically greater than or equal to the uniform distribution on

Figure - Graphical representation of general family-based graphical approach. Each vertex is associated with a family instead of a hypothesis, we term the graph as a family-based graph.
Global type I error rate control
The familywise error rate (FWER), which is the probability of incorrectly rejecting at least one true null hypothesis, is a commonly used notion of an overall measure of type I error when testing a single family of hypotheses.
Since we have multiple layers with any number of families within each layer, we consider this measure not locally for each family but globally. In other words, we define the overall FWER as the probability of incorrectly rejecting at least one true null hypothesis across all families of hypotheses for all layers.
"Layer" is an important concept in this paper! It represents the hierarchical structures over families instead of single hypothesis.
If it is bounded above by
Given the pre-specified
Remarks on error rate function introduced in Dmitrienko et al. (2008)
The error rate function was used to develop a simple stepwise approach for parallel gatekeeping strategies. In their discussion, the error rate function is required to be strictly less than

Note that in applications, if the error rate function
The definition of the error rate function in this section we used is a a little bit more general. For this function, the separability condition is not required when choosing local procedures for our suggested family-based graphical approach.
In the family-based approach, each family is tested by its own local procedure, thus it is associated with a particular error rate function
Let
Based on
Testing procedures and alpha transition approach
The procedures start with testing
The critical values used to locally test each family within the current layer is updated from its initially assigned value to one which incorporates certain portions of the critical values used in testing the families within the previous layers. This procedure stops testing when all families of the last layer
The distribution of the amount of critical values transferred among families can be pre-fixed by a transition coefficient set
Transition coefficient set G
Let
and
The
Algorithm
Algorithm 1 - Two layers with two families of hypotheses within each layer
Consider

Graph for two layer family-based procedure with m=4.
Step
Test family
Update the graph with
Step
Test
步骤简述:
此算法从第一层
的假设检验族 family 开始执行; 一旦
被检验, 的检验水准将基于转移矩阵 和error rate function 更新,并且 的更新将删除所有的与 相关的元素。 该算法的证明见原论文Appendix A.1。
Algorithm 2 - General multi-layer family-based graphical approach
The aforementioned two-layer four-family case demonstrates the inherent nature of sequential testing of the family-based graphical approach.
Now we generalize the graphical approach from two layers with two families of hypotheses in each layer to any
The general multi-layer family-based graphical approach is defined through the following algorithm:
Step
Test family
Update the graph with
Step
Test
简单案例:
考虑一个有
层 ,并且每层 只有一个假设检验族 的多重检验。 在多层family-based graphical approach下,
的初始检验水准是 ,如果 ;其余family分到的检验水准都是0。此时transition coefficient为 ,如果 。其他情况下 。 该算法的证明见原论文Appendix A.2。
对于上述算法,有以下几点额外的考量:
- 如果每个family都使用了能控制住FWER的local procedure,并且这些procedure都能满足separability condition (i.e. 每个local procedure的error rate function都严格小于
),那么这个算法等价于parallel gatekeeping中的general multistage gatekeeping procedure (Dmitrienko et al.(2008))。这个非常重要,表示truncated step-wise的传统检验步骤可以被图形化; - 如果每个family都使用了能控制住FWER的local procedure,并且这些procedure的error rate function的上确界为
,那么这个算法等价于特定的serial gatekeeping,比如传统Holm,fixed sequence procedure等; - 如果每个family都只有1个假设检验,其等价于fixed sequence procedure;
- 如果一个family内,null p-values之间的相关性已知,那么对于local procedures我们有更多选择,比如传统或者truncated Hochberg.
Example 1
Consider the Type II diabetes clinical trial example in Dmitrienko et al. (2007).
The trial compares three doses of an experimental drug (Doses L, M and H) versus placebo (Plac) with respect to one primary endpoint (P: Haemoglobin A1c), and two secondary endpoints (S1: Fasting serum glucose; S2: HDL cholesterol).
The three endpoints will be examined at each of the three doses, so a total of nine null hypotheses will be formulated and grouped into three families,
Family
- H vs Plac (
) - M vs Plac (
) - L vs Plac (
)
Similarly, family
The overall Type I error rate is pre-specified at

In this example, we assume that the primary endpoint P is more important than the secondary endpoints S1 and S2, thus F1 is always tested before testing F2 and F3.
Suppose that the secondary endpoints S1 and S2 are equally important, thus F2 and F3 are grouped into the same layer; the dose-placebo comparisons within each family are ordered a priori (H vs. Plac through L vs. Plac).
We choose the conventional fixed sequence procedure as local procedure for each family and the initial allocation of critical values for

The above figure visualizes this gatekeeping strategy. We start testing
Then, all of its local critical value 0.04 is equally assigned to
Finally, the testing results of Procedure are summarized in the above table. In addition, a figure below provides a graphical visualization for Procedure by using the hypothesis-based graphical approach.

Example 2
Consider an example as below

In hypothesis-based graphical approach, the initial weights for hypotheses
Suppose the raw p-value for these 6 hypotheses are

In the environment of R
, the gMCP
graph is
1 | m <- rbind(H1=c(0, 0.5, 0.5, 0, 0, 0), |
注意:
gMCP()
在RStudio环境中极易崩溃,需用原生R来运行
We can break down the MTP into steps as
Reject
and pass its level to andReject
, note that using the Algorithm 1 in Bretz et al.(2009) to update the weight.In this graph,
, the graph update algorithm is
其余权重更新具体数值可以根据这个公式计算
That is
This graph is updated to, and we can continuously reject
Reject
Finally both of
and can be rejected.
Could it be re-written as family-based graph approach?
Suppose
It is obvious that
1 | # Suppose H1 and H3 are not significant! |
After rejecting

由此,我们可以看出该检验的层次结构,
与 具有同等重要性,而 的检验需要基于 与 的检验结果。
The family-based graph looks like

Example 3 - Truncated Holm
Consider the 4 hypotheses truncated holm MTP (parallel gatekeeping) in hypothesis-based graph approach and the prespecified significance level is

Let
If
Similarly,

In family-based graphical approach,

where weight
Firstly, test hypotheses in
- If all hypotheses in
are tested and rejected using truncated Holm at initial , the since . Therefore, will be tested on level ; - If only one hypothesis in
is tested and rejected at initial , the since . That is, will be tested on level ; - If all hypotheses in
are tested and accepted using truncated Holm at initial , the . Therefore, will not be tested.
Reference
Bretz, Frank & Maurer, Willi & Brannath, Werner & Posch, Martin. (2009). A graphic approach to sequentially rejective multiple test procedures. Statistics in medicine. 28. 586-604. 10.1002/sim.3495.
Dmitrienko, Alex & Tamhane, Ajit & Wiens, Brian. (2008). General Multistage Gatekeeping Procedures. Biometrical journal. Biometrische Zeitschrift. 50. 667-77. 10.1002/bimj.200710464.
Qiu, Zhiying & Li, Yu & Guo, Wenge. (2018). A Family-based Graphical Approach for Testing Hierarchically Ordered Families of Hypotheses. 10.13140/RG.2.2.23109.29929.