Abstract
By adding a penalty to a candidate’s breeding value for its relationship with the selected individuals, two indexes were constructed as criteria for stepwise selection of superior individuals from populations with a hierarchical structure. The relationship was expressed in terms of either family contribution or group coancestry. One of the indexes was derived from an optimal selection model. A stepwise procedure that screened superior individuals one by one was introduced to make selection based on these indexes possible. Two penalty selection methods exclusively maximized gain at given coancestry. Both methods produced all identical solutions in most of the populations simulated, and were nearly equivalent in the remaining populations, particularly when heritability was high and the population structure was simple. A better balance between gain and coancestry following penalty index selection can be obtained by avoiding the two extreme solutions: combined- index and within-family selection, and using simple mating designs rather than complex ones.