Research in Mathematics ISSN: (Print) (Online) Journal homepage: https://www.tandfonline.com/loi/oama23 On the design of paired comparison experiments with application Eric Nyarko | To cite this article: Eric Nyarko | (2023) On the design of paired comparison experiments with application, Research in Mathematics, 10:1, 2180873, DOI: 10.1080/27684830.2023.2180873 To link to this article: https://doi.org/10.1080/27684830.2023.2180873 © 2023 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group. Published online: 05 Apr 2023. Submit your article to this journal Article views: 1241 View related articles View Crossmark data Full Terms & Conditions of access and use can be found at https://www.tandfonline.com/action/journalInformation?journalCode=oama23 RESEARCH IN MATHEMATICS 2023, VOL. 10, NO. 1, 2180873 https://doi.org/10.1080/27684830.2023.2180873 APPLIED & INTERDISCIPLINARY MATHEMATICS On the design of paired comparison experiments with application Eric Nyarko Department of Statistics and Actuarial Science, University of Ghana, Box LG 115, Legon, Accra, Ghana ABSTRACT ARTICLE HISTORY In practice, paired comparison experiments involving pairs of either full or partial profiles are Received 06 December 2021 frequently used. When all attributes have a general common number of levels, the problem of Accepted 10 February 2023 finding optimal designs is considered in the presence of a second-order interaction model. In this KEYWORDS setting, the D-optimal designs for the second-order interaction model have both types of pairs in full profile; interactions; which either all attributes have different levels or approximately half of the attributes are different. optimal design; paired The proposed optimal designs can be used as a benchmark to compare any design for estimating comparisons; partial profile; main effects and two- and three-attribute interactions. A practical situation that incorporates the profile strength corresponding second-order interactions is covered. 1 Introduction inclusion of interaction terms in design plans should be encouraged (Jaynes et al., 2016). Mandeville et al. (2014) Paired comparisons are related to experiments with two pointed out the importance of identifying main and options (alternatives). Such experiments are widely used interaction effects. By combining attributes, design in many fields of application, including health econom- plans can generate a rich source of data to evaluate real- ics, transportation economics, and marketing, to study life decision-making processes (Elrod et al., 1992; Kruk individual preferences toward new products or services et al., 2009; Shah et al., 2015; Soetevent & Kooreman, (Ryan, 2004; Sudhir et al., 2015), where behaviors of 2007). The present paper, where any of the three attri- interest typically involve quantitative responses butes interact, is motivated by the works mentioned (Scheffé, 1952). The present paper draws on this situa- above. tion of quantitative responses (so-called conjoint analy- Due to the limited cognitive ability to process infor- sis where responses are usually assessed on a rating mation in applications, a paired comparison task scale) as frequently encountered in marketing research including many attributes may result in respondent (Green et al., 2004; Rao, 2014; deBekker Grob Ew et al., decisions that do not reflect their actual preferences. 2012). A way to overcome these behaviors is to simplify the Typically, in paired comparison experiments, paired comparison task by holding the levels of some of respondents trade off one alternative against the other, the attributes constant in every pair. The profiles in such generated by an experimental design, and are described a pair are called partial profiles, and the number of by several attributes. Usually, one may consider experi- attributes allowed with potentially different levels in mental design, which allows estimating all the effects of the partial profiles is called the profile strength (e.g. see interest (typically main effects only or main effects plus Chrzan, 2010; Graßhoff et al., 2003; Green, 1974; Kessels some higher-order interaction effects) (Hoyos, 2010). et al., 2011). Jaynes et al. (2016) noted that the choice of the design In this paper, we mainly introduce an appropriate for choice experiments is critical because it determines model for the situation of full and partial profiles and which attributes’ effects and their interactions are iden- derive optimal designs in the presence of second-order tifiable. Analyzing higher-order interaction terms can (or three-attribute) interactions. We consider the case help explain welfare measures’ convergence or diver- when a general common number of level attributes gence. However, studies tend to ignore higher-order specifies the alternatives. A practical situation of interest interactions (Lancsar & Louviere, 2008), modelling incorporating the corresponding second-order interac- only main effects (Mogas et al., 2006). Therefore, the tions is also considered. In the statistical literature, work CONTACT Eric Nyarko Email ericnyarko@ug.edu.gh Department of Statistics and Actuarial Science, University of Ghana, Box LG 115, Legon, Accra, Ghana Reviewing editor: Guohua Zou University of the Chinese Academy of Sciences, China © 2023 The Author(s). This open access article is distributed under a Creative Commons Attribution (CC-BY) 4.0 license. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. The terms on which this article has been published allow the posting of the Accepted Manuscript in a repository by the author(s) or with their consent. 2 E. NYARKO on determining optimal designs for binary attributes has direct observation, in paired comparison experiments, been investigated (Berkum, 1987; Schwabe et al., 2003) the utilities for the alternatives are usually not directly in the case of full and/or partial profiles in additive as observed. Only observations Ynði; jÞ ¼ Y~nðiÞ Y~nðjÞ well as two-attribute interaction setup. Corresponding are available for comparing pairs ði; jÞ of alternatives i results when the common number of the attribute levels and j which are chosen from the design region is larger than two have been obtained (see Graßhoff X ¼ I � I . In that case, the utilities for the alternatives et al., 2003). Here, we treat the case of three-attribute are properly described by the linear paired comparison interactions when the common number of the attribute model levels is at least two and provides some detailed insights. T The two-level situation has been investigated in the case Ynði; jÞ ¼ ðfðiÞ fðjÞÞ βþ εn; (2) of both full and partial profiles (Nyarko & Schwabe, where fðiÞ fðjÞ is the derived regression function and 2019). the random errors εnði; jÞ ¼ ~εnðiÞ ~εnðjÞ associated The remainder of the paper is organized as follows. with the different pairs ði; jÞ are assumed to be uncorre- A general model is introduced in Section 2 for linear lated with constant variance and zero mean. Here, the paired comparisons related to experiments with two block effects μn are immaterial. options. Three-attribute interaction model for full and The performance of the statistical analysis based on partial profiles is presented in Section 3, and optimal a paired comparison experiment depends on the pairs in designs are characterized in Section 4. Theoretical the presented preference task. The choice of such pairs design constructions are presented in Section 5, ði1; j1Þ; . . . ; ðiN ; jNÞ is called a design of size N. The a practical situation of interest is considered in quality of such a design is measured by its information Section 6, and the final Section 7 offers some matrix. conclusions. This article considers approximate designs � (e.g. see Kiefer, 1959) which are defined as discrete probability 2 General setting measures on the design region X of all pairs ði; jÞ. Moreover, every approximate design � which assigns In any experimental situation, the experiment’s out- only rational weights �ði; jÞ to all pairs ði; jÞ in its sup- come depends on some factors (attributes), say, K of port points can be realized as an exact design �N of size influence. In this setting, the dependence can be N consisting of the pairs ði1; j1Þ; . . . ; ðiN ; jNÞ. Note that described by a vector of regression functions f . In for an exact design �N the normalized information what follows, we define a single alternative by i ¼ matrix Mð�NÞ coincides with the information matrix ði1; . . . ; iKÞ where ik is the component of the Kth attri- Mð�Þ of the corresponding approximate design �. bute, k ¼ 1; . . . ;K. Any utility (not observe) Y~nðiÞ of the Optimality criteria for approximate designs � are single alternative i ¼ ði1; . . . ; iKÞ subject to a block effect functionals of Mð�Þ. As a scalar measure of design μn and a random error ~εn, which is assumed to be quality, here we consider the criterion of D-optimality. uncorrelated with constant variance and zero mean, An approximate design �� is D-optimal if it maximizes can be formalized by a general linear model the determinant of the information matrix, that is, if T detMð��Þ � detMð�Þ for every approximate design �Y~nðiÞ ¼ μn þ fðiÞ βþ ~εn; (1) on X . where the index n denotes the nth presentation, n ¼ 1; . . . ;N, and the alternative i is chosen from a set I ¼ f1; . . . ; v Kg . Here, the vector of known regression 3 Second-order interaction model functions f describes the form of the functional relation- In applications, one may be interested in the utility ship between the alternative i and the corresponding estimates of the main effects and interactions between mean response E TðY~nðiÞÞ ¼ μn þ fðiÞ β, and β is the the levels of the attributes. For that setting, optimal unknown parameter vector of interest. Usually, to designs have been derived (van Graßhoff et al., 2003; make statistical inferences on the unknown parameters, Berkum Eem, 1987) in a two-attribute interaction setup. several pairs are presented to get rid of the influence of This paper considers a three-attribute interaction the block effect μn due to a variety of unobservable model. Corresponding results for the particular case of influences. The actual differences of the latent utilities binary attributes have been obtained (Nyarko & are observed for the alternatives presented in a pair. Schwabe, 2019). More specifically, unlike in standard experimental Analogous to Nyarko and Schwabe (2019), we first designs where there is a possibility of only a single or start with the situation of full profiles. In that case, RMS: RESEARCH IN MATHEMATICS & STATISTICS 3 each alternative is represented by level combinations Kðv 1Þ parameters, the second components fði1Þ � in which all attributes are involved. For such alterna- fði2Þ; . . . ; fðiK 1Þ � fðiKÞ are associated with the two- tives, we denote by i ¼ ði1; . . . ; iKÞ and j ¼ ðj1; . . . ; jKÞ attribute interactions and have p2 ¼ ð1=2ÞKðK the first alternative and the second alternative, respec- 1 2 Þðv 1Þ parameters, and the remaining components tively, which are both elements of the set I ¼ fði1Þ � fði2Þ � fði3Þ; . . . ; 1 v K f ; . . . ; g where 1 and v represent the first and last fðiK 2Þ � fðiK 1Þ � fðiKÞ are associated with the three- level of each kth component, k ¼ 1; . . . ;K. Here ði; jÞ attribute interactions and have p3 ¼ ð1=6Þ is an ordered pair of alternatives i and j which is KðK 1ÞðK 2Þðv 1 3 Þ parameters. chosen from the design region X ¼ I � I . Note that As was already pointed out, because of the limited for each attribute k, the corresponding regression cognitive ability to process information, a preference functions fk ¼ f coincide with the one-way layout task including many attributes may enhance respon- (see Graßhoff et al., 2003). dent decisions that do not reflect their actual prefer- In the presence of up to three-attribute interactions, ences. A way to overcome this problem is to use direct responses Y~n at alternative i ¼ ði1; . . . ; iKÞ can be partial profiles. In a partial profile, every pair consists modeled as of alternatives described by a predefined number S of XK X attributes. The same attributes are used throughout Y~nðiÞ ¼ μn þ f i T ð kÞ β Tk þ ðfðikÞ � fði,ÞÞ βk both alternatives within a pair but with potentially , k¼1 k< , different levels. In contrast, the remaining K S X þ ðf i Tð kÞ � fði Þ � fðimÞÞ β þ ~εn; attributes are not shown and remain thus unspeci-, k,m k< , 0 and, by symmetry, obtained by first calculating the values of h3ðdÞ and the value of h3 at the local maximum d3;max ¼ determining the maximum. It is worthwhile men- pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiðSv S vþ 2Þ=v 9Svþ 3v2 9S 18vþ 18=ð3vÞ tioning that for very moderate values of v (v ¼ 2, is equal to h3ðd3;maxÞ< h3ðSÞ which proves that the global for example) the optimal comparison depth d�2 ¼ S maximum of h3 is attained at d � S for 0 � d � S. but this is not true for the case when S ¼ K ¼ 3. For invariant designs ��, the value of the variance Moreover, for sufficiently large values of v (v ¼ 20, function evaluated at comparison depth d may be for example) the optimal comparison denoted by Vðd; ��Þ, say, where Vðd; ��Þ ¼ Vðði; jÞ; ��Þ depth d�2 ¼ S 2. on ðSÞXd . It can be shown that in this case, the variance function Vðd; ��Þ of the invariant design �� has the fol- Theorem 1 ðaÞ For S ¼ 3 the uniform design � lowing structure.�1 is D- optimal for the vector of three-attribute interaction effects ðβ Tk,mÞk< , 0 for ðfðikÞ f Tðj 1kÞÞ M ðfðikÞ fðjkÞÞ ¼ v 1: (13) RMS: RESEARCH IN MATHEMATICS & STATISTICS 7 Further, for the regression function associated with the interaction terms for which ðiki,imÞ and ðjkj,jmÞ differ two-attribute interactions of the attributes k and ,, say, in exactly two of the associated three attributes and we obtain finally, there are 12 ðS dÞðS d 1Þd three-attribute interaction terms for which ðiki,imÞ and ðjkj,jmÞ differ ðfðikÞ � f i Tð ,Þ fðj Þ � fðj ÞÞ M 1k , �M 1ðfðikÞ � fði,Þ fðjkÞ � fðj,ÞÞ in exactly one of the associated three attributes. As a consequence, we obtain T T T T ¼ fði Þ M 1k fðikÞ � fði Þ M 1, fði,Þ þ fðj Þ M 1fðj Þ � fðj Þ M 1k k , fðj,Þ V T 1ðd; ��Þ ¼ ðfðiÞ fðjÞÞ Mð��Þ ðfðiÞ fðjÞÞ f T 2 ði Þ M 1fðj Þ � f Tði Þ M 1fðj TÞ fðj Þ M 1fði Þ dðv 1Þ dðd 1Þ ðv 1Þ ðv 2Þk k , , k k ¼ þ f T h1ð� �Þ 2 2vh2ð��Þ � ðj,Þ M 1fði,Þ ðv 1 3Þ þ dðS dÞ ( 2 2vh2ð��Þ ðv 1Þ ðv 2Þ 2v for ik�jk;i,�j,¼ (14) dðd 1 d 2 3 Þð Þ ðv 1Þ ðv2 3vþ 3Þ ðv 3 1Þ þ 2v for i 2 k�jk;i �,¼j, or ik¼jk;i,�j,: 6 4v h3ð�Þ ðS 4 dÞdðd 1Þ ðv 1Þ ðv 2Þ Accordingly, for the regression function associated with þ 2 4v2h ð��Þ the interaction of the attributes k, , and m, say, we 3 5 obtain ðS dÞðS d 1Þd ðv 1Þþ 2 4v2h3ð��Þ d 2ðv 1Þ dðv 1Þ f i f i f i f j f j f j TM 1 M 1 M 1 ¼ þ ð2Sv 2S dv vþ 2Þð ð kÞ � ð ,Þ � ð mÞ ð kÞ � ð ,Þ � ð mÞÞ � � h1ð��Þ 4vh2ð��Þ dðv 1 3Þ þ ð3S2v22 6S 2v 6Sv2 þ 3S2 3Sdv2 þ 3Sdv � � ðfðikÞ � f i f i f j f j f j 24v h3ð�Þð ,Þ � ð mÞ ð kÞ � ð ,Þ � ð mÞÞ þ 3dv2 þ 15Sv 9Sþ d2v2 6dv þ 2v2 6vþ 6Þ; ¼ fði T 1 T 1 T 1kÞ M fðikÞ � fði,Þ M fði,Þ � fðimÞ M f ðimÞ for ði; j ðSÞ Þ 2 Xd which proves the proposed formula. In the case of a single comparison depth, it can be shown that the corresponding invariant design ��has the following f j TM 1f j f j Tþ ð kÞ ð kÞ � ð ,Þ M 1f Tðj,Þ � fðj Þ M 1m fðjmÞ structure. It is worth noting that if d d0¼ , then by the Kiefer-Wolfowitz equivalence theorem (Kiefer & Wolfowitz, 1960) the variance function Vðd; �� T T T d 0 Þ will be fði Þ M 1k fðj 1kÞ � fði,Þ M fðj,Þ � fðimÞ M 1fðjmÞ equal to the total number of the model para- meters, p ¼ p1 þ p2 þ p3. f j TM 1f i T T ð kÞ ð kÞ � fðj Þ M 1, fði,Þ � fðj Þ M 1m fðimÞ Corollary 1 For a uniform design ��d0 on a single com-8 v 1 3> ð Þ ðv2 3vþ3Þ for i j parison depth d 0 the variance function is given by < 4v2 k� k; i,�j,;im�jm ¼ ðv 4 � � 1Þ ðv 2Þ 4v2 for ik�jk;i,�j,;im¼jm (15) Vðd; �� Þ ¼ d 2Sv 2S dv vþ2 λðdÞ> d0 d0 p1 þ p2: v 5 2Sv 2S d 0v v 2þ p þ 3 λ 0 :ðd Þ ð 1Þ 4v2 for ik�jk;i,¼j,;im¼jm: The following result gives an upper bound on the num- Now for a pair of alternatives i j ðSÞð ; Þ 2 X d of compar- ber of comparison depths required for a D-optimal design. ison depth d: there are exactly d attributes of the main effects for which ik and jk differ, there are 12 dðd 1Þ two-attribute interaction terms for which ðiki,Þ and Theorem 3 The D-optimal design �� for the three- ðjkj,Þ differ in all two attributes k and ,, there are dðS attribute interaction model is supported on, at most, dÞ two-attribute interaction terms for which ðiki,Þ and three different comparison depths S, d�, and d� þ 1. ðjkj,Þ differ in exactly one attribute k or ,, there are 1 6 dðd 1Þðd 2Þ three-attribute interaction terms for Proof of Theorem 3. According to a corollary of which ðiki,imÞ and ðjkj,jmÞ differ in all three attributes k, Kiefer-Wolfowitz equivalence theorem (Kiefer & , and m, there are 1 ðS dÞdðd 1Þ three-attribute Wolfowitz, 1960, p. 364) for the D-optimal design 2 8 E. NYARKO ���, the variance function Vðd; ���Þ is equal to the ln det �ðMð�� ;w�SÞÞ ¼ cþ Kðv 1Þ � lnðSv Sþ Sw � S number of parameters p for all D. By Theorem 2, þ 2w�v 3w� 2vþ 3Þ the variance function is a cubic polynomial in the S S 2 comparison depth D with a positive leading coeffi- KðK 1Þðv 1Þ � ln λ1þ cient. The variance function Vðd; ���Þ may thus be 2 3 equal to p for, at most, three different values KðK 1ÞðK 2Þðv 1Þ � ln λ2þ ; d1 < d2 < d3 of D, say. Now, by the Kiefer-Wolfowitz 6 equivalence theorem (Kiefer & Wolfowitz, 1960) where c is a constant independent of the weight w�S . itself Vðd; ���Þ � p for all d ¼ 0; 1; . . . ; S. Taking derivatives with respect to w�S , we obtain We now make use of the analytic tool of the Kiefer- @ � Wolfowitz equivalence theorem to establish the D- ln detðMð�� ;w� @w� S ÞÞ S optimality of the design ��� by direct maximization of � �Sþ 2v 3 lnðdetðMðw� �� � �d�� d� þ ð1 wd�Þ�SÞÞÞ (e.g. see Kiefer & ¼ Kðv 1Þ � Sv Sþ Sw�S þ 2w�Sv 3w�S 2vþ 3 Wolfowitz, 1960). K K 1 v 1 2ð Þð Þ � þ � S2 Svþ 2Sþ 2v2 5vþ 3 By Lemma 1 the entries of the information matrix 2λ1 3 Mð���;w�SÞ are specified by KðK 1ÞðK 2Þðv 1Þþ � 6λ2 h �� w� w�h S 1 w� � Sv SþSw �þ2w�S S v 3w � S 2vþ3 � 1ð� ; SÞ ¼ S 1ð Þ þ ð SÞh1ðd Þ ¼ Kv ; S3 4Sv2 3S2 3Sþ 9Svþ 6v2 15vþ 9 h �� w� w�h K 1 w� h d� λ1 which has root w � S ¼ 1 w�d� . This root gives 2ð� ; SÞ ¼ S 2ð Þ þ ð SÞ 2ð Þ ¼ 2KðK 1 ;Þv2 a maximum for the determinant. The design ��� is thus where D-optimal when we consider the particular case of the 2 2 2 2 2 2 reduced design region X S [ Xd� .λ1 ¼ S v 2S v Sv S w� Sw�vþ 2w�S S Sv Again, by inserting the corresponding functions þ S2 þ 3Svþ 2Sw� 2v2 5w�S Sv h ð���;w�Þ; h ð���;w�1 S 2 SÞ and h3ð���;w�SÞ into the represen- 2Sþ 5vþ 3w�S 3; tation of the variance function VðS; d; ���;w�SÞ in and Theorem 2, we obtain VðS; d�; ���;w�SÞ ¼ VðS; d �; d� þ 1; ���;w�SÞ � p: h � � � � � λ23ð�� ;wSÞ ¼ wSh3ðSÞ þ ð1 wSÞh3ðd Þ ¼ 4KðK 1ÞðK 2 v3 ; Þ Hence, S; d� and d� þ 1 are integer solutions for the λ ¼ S3v3 3S3v2 3S2v3 þ 3S3vþ S3w� þ 9S2v2 maximum of the variance function, which shows the 2 S 3 � 2 3 2 2 � 2 D-optimality of the design because of the equivalence þ 2Sv 4SwSv S 9S v 3S wS 2Sv theorem by Kiefer and Wolfowitz (1960). þ 9Sw�Svþ 6w � 2 2 � 2 Sv þ 3S 3Sv 3SwS 6v For the results on parts of the parameter vector 15w�vþ 3Sþ 15vþ 9w�S S 9: involving main effects up to three-attribute interaction, the D-optimal design for the complete parameter vector Now, since the determinant of the information matrix p p may depend on the profile strength S as can be seen by Mð���;w�SÞ is proportional to h � � 1 1ð�� ;wSÞ h2ð���;w� 2 SÞ h3 the numerical examples for the case of arbitrary levels, p ð���;w� 3SÞ , we thus obtain v � 2 presented in Table 2. We note that for the case Table 2. Optimal designs with intermediate comparison depths d� in boldface and optimal weights w�d� of the form ðd �, w�d� Þ for the case of full profiles ðS ¼ KÞ and v-levels v K 2 3 4 5 6 7 8 4 (2, 0.857) 2 2 2 2 2 2 5 (2, 0.833) (2, 0.667) 3 3 3 3 3 6 (3, 0.732) (3, 0.789) 3 4 4 4 4 7 (3, 0.697) (4, 0.322) 4 4 4 5 5 8 (3, 0.644) 4 (5, 0.425) 5 5 5 5 9 (4, 0.577) 5 5 6 6 6 6 10 (4, 0.538) 5 6 6 7 7 7 Source: The author. RMS: RESEARCH IN MATHEMATICS & STATISTICS 9 v ¼ 2, S ¼ K ¼ 3 of full profiles and complete interac- to specify an explicit formula for calculating d�, the fol- tions, the D-optimal design given explicitly in Nyarko lowing Table 2 shows the corresponding optimal designs and Schwabe (2019) indicates that all three comparison with their optimal comparison depths d� in boldface and depths are needed for D-optimality. their corresponding weights w�d� for various choices of For S � 4, numerical computations indicate that at attributes K between 4 and 10 and levels v ¼ 2; . . . ; 8. most two different comparison depths S and d� may be Entries of the form ðd�;w�d� Þ indicate that invariant required for D-optimality. Because it is generally difficult designs ��� ¼ w� ��� � �d� d� þ ð1 wd� Þ�S have to be Table 3. Values of the variance function Vðd; ���Þ for ��� from Table 2 in the case of full profiles (S ¼ K) and v-levels (boldface 1 corresponds to the optimal comparison depths d�) d K v 1 2 3 4 5 6 7 8 9 10 4 2 0.875 1 0.875 1 3 0.813 1 0.938 1 4 0.793 1 0.953 0.983 5 0.783 1 0.962 0.980 6 0.777 1 0.968 0.980 7 0.773 1 0.973 0.981 8 0.770 1 0.976 0.982 5 2 0.760 1 0.960 0.880 1 3 0.723 1 1 0.954 1 4 0.689 0.967 1 0.952 0.987 5 0.666 0.951 1 0.961 0.981 6 0.653 0.941 1 0.968 0.980 7 0.644 0.934 1 0.972 0.981 8 0.638 0.929 1 0.976 0.982 6 2 0.701 0.983 1 0.906 0.855 1 3 0.624 0.921 1 0.968 0.932 1 4 0.591 0.895 1 0.993 0.963 0.997 5 0.576 0.882 0.997 1 0.972 0.992 6 0.560 0.865 0.987 1 0.976 0.989 7 0.550 0.854 0.981 1 0.979 0.988 8 0.543 0.846 0.977 1 0.982 0.988 7 2 0.615 0.917 1 0.956 0.879 0.863 1 3 0.553 0.860 0.988 1 0.963 0.941 1 4 0.519 0.822 0.965 1 0.981 0.962 0.997 5 0.498 0.800 0.952 1 0.992 0.974 0.993 6 0.487 0.787 0.944 1 0.999 0.983 0.995 7 0.479 0.777 0.937 0.997 1 0.985 0.994 8 0.471 0.768 0.929 0.994 1 0.987 0.993 8 2 0.559 0.872 1 1 0.945 0.884 0.884 1 3 0.490 0.792 0.948 1 0.990 0.958 0.948 1 4 0.462 0.759 0.924 0.993 1 0.980 0.969 1 5 0.442 0.732 0.902 0.981 1 0.988 0.977 0.995 6 0.429 0.716 0.889 0.974 1 0.994 0.982 0.994 7 0.421 0.706 0.880 0.970 1 0.997 0.987 0.995 8 0.415 0.698 0.874 0.960 1 1 0.991 0.996 9 2 0.504 0.811 0.962 1 0.969 0.910 0.868 0.883 1 3 0.437 0.726 0.894 0.972 1 0.969 0.946 0.946 1 4 0.414 0.696 0.872 0.965 1 0.994 0.977 0.971 1 5 0.397 0.674 0.853 0.953 0.995 1 0.989 0.981 1 6 0.384 0.657 0.836 0.940 0.989 1 0.992 0.985 0.996 7 0.376 0.645 0.825 0.932 0.985 1 0.995 0.988 0.995 8 0.370 0.637 0.817 0.927 0.982 1 0.997 0.990 0.996 10 2 0.462 0.763 0.932 1 0.997 0.956 0.905 0.874 0.896 1 3 0.395 0.669 0.843 0.938 1 0.972 0.953 0.938 0.947 1 4 0.374 0.642 0.822 0.929 0.981 1 0.987 0.974 0.972 1 5 0.359 0.622 0.803 0.917 0.977 1 1 0.989 0.985 1 6 0.348 0.606 0.786 0.903 0.968 0.996 1 0.993 0.988 0.998 7 0.340 0.594 0.774 0.892 0.961 0.993 1 0.995 0.990 0.997 8 0.335 0.586 0.765 0.885 0.956 0.990 1 0.996 0.991 0.996 Source: author. As was already pointed out, the corresponding designs possess many comparisons. For the case of binary attributes (v ¼ 2) the designs in Table 2 when K ¼ 4; 5 and 6, for instance, consist of 96; 320 and 1280 pairs, respectively. In this situation, it is possible to construct a design for estimating main effects and two and three-attribute interactions. For example, if K ¼ 3, v ¼ 2 and d� ¼ 1, the paired comparison design consists of 24 pairs. In the next section, algorithms for generating such designs are presented. It is worth noting that the design contains repeated pairs, which is a result of a large number of comparisons. This design is used as an example to assess university students’ satisfaction with online teaching and psychological pressure on learning during the COVID-19 pandemic later on. 10 E. NYARKO considered, while for single entries d� the optimal design Table 4. Design for estimating main effects and two- and three- ��� ¼ ��� has to be considered which is uniform on the attribute interactions of K ¼ 3 two-level attributesd� optimal comparison depth d�. In particular, for the case n Pair ðin; jnÞ S ¼ K ¼ 4 of full profiles where v 2 the corresponding 1 ((1,1,1,1,1,1,1), (2,1,1,2,2,1,2))¼ 2 ((1,2,1,2,1,2,2), (2,2,1,1,2,2,1)) invariant design depends on the optimal comparison 3 ((1,1,1,1,1,1,1), (2,1,1,2,2,1,2)) depths d� 2 and S 4 with optimal weights w� 4 ((1,2,1,2,1,2,2), (2,2,1,1,2,2,1))¼ ¼ d� ¼ 5 ((1,1,2,1,2,2,2), (2,1,2,2,1,2,1)) 0:857 and w�d� ¼ 0:143, respectively. The optimality of 6 ((1,2,2,2,2,1,1), (2,2,2,1,1,1,2)) the designs presented in Table 2 has been checked 7 ((1,1,2,1,2,2,2), (2,1,2,2,1,2,1))8 ((1,2,2,2,2,1,1), (2,2,2,1,1,1,2)) numerically under the equivalence theorem. The values 9 ((1,1,1,1,1,1,1), (1,2,1,2,1,2,2)) of the normalized variance function V d �� p are 10 ((1,1,2,1,2,2,2), (1,2,2,2,2,1,1))ð ; � Þ= 11 ((1,1,1,1,1,1,1), (1,2,1,2,1,2,2)) recorded in Table 3, where maximal values less than or 12 ((1,1,2,1,2,2,2), (1,2,2,2,2,1,1)) equal to 1 establish optimality. 13 ((2,1,1,2,2,1,2), (2,2,1,1,2,2,1)) 14 ((2,1,2,2,1,2,1), (2,2,2,1,1,1,2)) 15 ((2,1,1,2,2,1,2), (2,2,1,1,2,2,1)) 16 ((2,1,2,2,1,2,1), (2,2,2,1,1,1,2)) 5 Algorithmic design approach 17 ((1,1,1,1,1,1,1), (1,1,2,1,2,2,2)) 18 ((2,1,1,2,2,1,2), (2,1,2,2,1,2,1)) This section provides a theoretical construction method for 19 ((1,1,1,1,1,1,1), (1,1,2,1,2,2,2)) the corresponding optimal designs. The construction uses 20 ((2,1,1,2,2,1,2), (2,1,2,2,1,2,1))21 ((1,2,1,2,1,2,2), (1,2,2,2,2,1,1)) an algorithmic design approach (Großmann et al., 2012; 22 ((2,2,1,1,2,2,1), (2,2,2,1,1,1,2)) Nyarko & Doku-Amponsah, 2022) to create an exact 23 ((1,2,1,2,1,2,2), (1,2,2,2,2,1,1))24 ((2,2,1,1,2,2,1), (2,2,2,1,1,1,2)) design with N pairs. Let �N;d be an exact design with Source: author. attributes K that differ in d (so-called comparison depth), allowing estimation of main effects and two and three attribute interactions. The algorithm generates two N � to be presented to subjects differs in only one attribute K matrices I and J from the design region X with treat- (d ¼ 1). In an experimental situation, suppose d ment combinations i ; . . . ; i and j ; . . . ; j , respectively. a researcher is interested in constructing a design �1 N N;d 1 N For given K and d, the method requires three building with N ¼ 24 pairs that differ in only one attribute to blocks involving an m� ðK dÞ matrix F which repre- compare products with K ¼ 3 attributes. For this situation, sents a regular two-level factorial design of resolution III or the pairs ðin; jnÞ for n ¼ 1; . . . ; 24 with orthogonal coded higher for orthogonal coded (i.e. � 1) K d binary attri- levels � 1, where the orthogonal coded first and last level butes, a Hadamard matrix H of order t � d, and a matrix B of each attribute are assigned with the actual levels 1 and 2, of dimension a d � b, which represents a balanced incom- respectively (see Table 4). A Hadamard matrix of order plete block design for K treatments k ¼ 1; . . . ;K in b t ¼ 1, a regular 23 full factorial design, and an incomplete blocks of size d. These building blocks are used to construct block design with blocks f1g; f2g, and f3g were used to designs �N d with N ¼ bmt treatment combinations generate the design. The levels of attribute 1 in each alter-; selected from X . The following is an illustration of the native are determined by a column from the combined d construction: rows of the Hadamard matrix and the regular 23 full Step 1: Let A be a t � d matrix obtained by selecting factorial design in the 1 8 pairs. The levels of the attri- d columns from H and let F be an m� ðK dÞ matrix butes in columns 2 and 3 are determined by the corre- representing the regular factorial design. sponding column of the combined rows of the Hadamard Step 2: Combining the rows of A and F yields the matrix and the regular factorial design for pairs 9 16 and mt � K matrix I. The matrix J is obtained similarly by 17 24, respectively, while the levels of the other remain- using A. ing attributes are the same in both alternatives and depend Step 3: Rearrange the columns of I and J based on on the regular 23 full factorial design. a permutation derived from the first b blocks of B. More It is worth noting that for given values of K and d, and specifically, the design’s mt pairs are obtained by combin- by selecting a regular two-level full factorial design F with ing every row of the permuted matrix of I with the same m treatment combinations and a Hadamard matrix H of row of the permuted matrix of J (see Großmann & appropriate order d, similar designs can be constructed Schwabe, 2015; Großmann et al., 2012). This procedure is with N ¼ bmd pairs of the aforementioned type by per- repeated for each of the remaining b columns of B. The forming a computer search (e.g. see Großmann & final design has N ¼ bmt treatment combinations. Schwabe, 2015) over appropriately selected balanced For illustrative purposes, we construct a design to com- incomplete block design B with K treatments k ¼ pare products with K ¼ 3 attributes, where the choice task 1; . . . ;K in b blocks of size d. RMS: RESEARCH IN MATHEMATICS & STATISTICS 11 Figure 1. Effect summary of students’ satisfaction with online teaching during the COVID-19 pandemic. Figure 2. Effect summary of students experiencing little psychological pressure during COVID-19 pandemic. Figure 3. Effect summary of students experiencing lots of psychological pressure during COVID-19 pandemic. 12 E. NYARKO Figure 4. Effect summary of students experiencing no psychological pressure during COVID-19 pandemic. 6 Application Acknowledgements This section considers a practical situation where up to This work was partially supported by Grant – Doctoral three-factor interaction is interesting. In particular, we Programmes in Germany, 2016=2017ð57214224Þ – of the employ the design presented in Table 4 to assess university German Academic Exchange Service (DAAD). students’ satisfaction with online teaching and psychologi- cal pressure on learning during the COVID-19 pandemic. Disclosure statement The JMP Pro (Version 16.0) statistical software was used to analyze the responses of 150 students of the University of No potential conflict of interest was reported by the author. Ghana who were intercepted on the university campus. The full sample results are presented (Figure 1). These results were further classified according to levels of psycho- Public interest statement logical pressure (Figures 1–4). The Log-Worth and the corresponding P-values of the various factors (or attri- In modern scientific experiments, paired comparison butes) are also reported in ascending order of importance. experiments involving pairs of either full or partial profiles are frequently used. Typically, one may be interested in It can be observed that, in most cases, the three-factor the main effects as well as interactions between the attri- interactions perform well and are important. butes. Accordingly, we introduce an appropriate model for full and partial profiles and derive optimal designs that allow estimating all the main effects plus two plus three attribute interaction effects of interest with real-life 7 Discussion applications. For paired comparisons where the alternatives are described by an analysis of variance model with three- Notes on contributor attribute interactions, optimal designs require that pairs should be considered in which either all attributes have Eric Nyarko is a Statistics Lecturer distinct levels or approximately a portion of the attributes at the University of Ghana, Accra, Ghana. He received M.Phil. from are distinct, and the remaining portion of the attributes the University of Ghana, Ghana, in coincides to obtain a D-optimal design for the whole para- 2014 and a Ph.D. degree from the meter vector. Optimal designs may be concentrated on University of Magdeburg, Mag one, two, or three different comparison depths depending deburg, Germany, in 2019. His on the number of levels and attributes. The so obtained research interests include optimal design of experiments, design and approximate and exact designs can serve as a benchmark to analysis of machine learning judge the efficiency of any competing design. A practical experiments, discrete choice situation of interest is explored where the design identifies experiments, conjoint analysis, main effects and two- and three-attribute interactions. and best-worst scaling and its application. RMS: RESEARCH IN MATHEMATICS & STATISTICS 13 Funding Kessels, R., Jones, B., & Goos, P. (2011). Bayesian optimal designs for discrete choice experiments with partial The work was supported by the Deutscher Akademischer profiles. Journal of Choice Modelling, 4, 52–74. https://doi. Austauschdienst [57214224] org/10.1016/S1755-5345(13)70042-3 Kiefer, J. (1959). Optimum experimental designs. Journal of the Royal Statistical Society, Series B, 21(2), 272–304. ORCID https://doi.org/10.1111/j.2517-6161.1959.tb00338.x Eric Nyarko http://orcid.org/0000-0002-1666-7489 Kiefer, J., & Wolfowitz, J. (1960). The equivalence of two extremum problems. Canadian Journal of Mathematics, 12, 363–366. https://doi.org/10.4153/CJM-1960-030-4 References Kruk, M. E., Paczkowski, M., Mbaruku, G., De Pinho, H., & Galea, S. (2009). Women’s preferences for place of delivery Berkumvan, E. E. M. (1987). Optimal Paired Comparison in rural Tanzania: A population-based discrete choice Designs for Factorial Experiments CWI Tract 31. Centrum experiment. American Journal of Public Health, 99(9), voor Wiskunde en Informatica. 1666–1672. https://doi.org/10.2105/AJPH.2008.146209 Chrzan, K. (2010). Using partial profile choice experiments to Lancsar, E., & Louviere, J. (2008). Conducting discrete choice handle large numbers of attributes. International Journal of experiments to inform healthcare decision making. Market Research, 52, 827–840. https://doi.org/10.2501/ Pharmacoeconomics, 26, 661–677. https://doi.org/10.2165/ S1470785310201673 00019053-200826080-00004 deBekker Grob Ew, Ryan, M., Gerard, K., & de Bekker-Grob, Mandeville, K. L., Lagarde, M., & Hanson, K. (2014). The use of E. W. (2012). Discrete choice experiments in health eco- discrete choice experiments to inform health workforce policy: nomics: A review of the literature. Health Economics, 21(2), A systematic review. BMC Health Services Research, 14, 1–14. 145–172. https://doi.org/10.1002/hec.1697 https://doi.org/10.1186/1472-6963-14-367 Elrod, T., Louviere, J. J., & Davey, K. S. (1992). An empirical Mogas, J., Riera, P., & Bennett, J. (2006). A comparison of comparison of ratings-based and choice-based conjoint contingent valuation and choice modelling with models. Journal of Marketing Research, 29, 368–377. second-order interactions. Journal of Forest Economics, 12, https://doi.org/10.1177/002224379202900307 5–30. https://doi.org/10.1016/j.jfe.2005.11.001 K Graßhoff, U., Großmann, H., Holling, H., & Schwabe, R. Nyarko, E. (2020) Optimal 2 paired comparison designs for (2003). Optimal paired comparison designs for first-order third-order interactions. Statistical Papers https://doi.org/ interactions. Statistics, 37, 373–386. https://doi.org/10. 10.1007/s00362-020-01174-4 1080/0233188031000154812 Nyarko, E., & Doku-Amponsah, K. (2022). Approximate and Green, P. E. (1974). On the design of choice experiments invol- exact optimal designs for paired comparison experiments. ving multifactor alternatives. The Journal of Consumer Calcutta Statistical Association Bulletin, 74(1), Research, 1, 61–68. https://doi.org/10.1086/208592 0 0 0 8 0 6 8 3 2 2 1 0 7 9 9 6 5 . h t t p s : / / d o i . o r g / 1 0 . 1 1 7 7 / Green, P. E., Krieger, A. M., & Wind, Y. (2004). Thirty years of 00080683221079965 conjoint analysis: Reflections and prospects. In Wind Y, Nyarko, E., & Schwabe, R. (2019). Optimal designs for Green P, editors. Marketing research and modeling: Progress second-order interactions in paired comparison experiments and prospects (pp. 117–139). Springer. with binary attributes. Journal of Statistical Theory and Practice Großmann, H. (2017). Partial-profile choice designs for esti- 13, 13(4). https://doi.org/10.1007/s42519-019-0051-x mating main effects and interactions of two-level attributes Rao, V. R. (2014). Applied Conjoint Analysis. Springer. https:// from paired comparison data. Journal of Statistical Theory doi.org/10.1007/978-3-540-87753-0 and Practice, 11, 236–253. https://doi.org/10.1080/ Ryan, M. (2004). Discrete choice experiments in health care. The 15598608.2016.1197868 BMJ, 328(7436), 360–361. https://doi.org/10.1136/bmj.328. Großmann, H., & Schwabe, R. (2015). Design for discrete 7436.360 choice experiments. In A. Dean, M. Morris, J. Stufken, & Scheffé, H. (1952). An analysis of variance for paired D. Bingham (Eds.), Handbook of Design and Analysis of comparisons. Journal of the American Statistical Experiments (pp. 787–831). Chapman and Hall/CRC. Association, 47, 381–400. https://doi.org/10.1080/ 01621459.1952.10501179 Großmann, H., Schwabe, R., & Gilmour, S. G. (2012). Designs for Schwabe, R. (1996). Optimum designs for multi-factor models. first-order interactions in paired comparison experiments with Springer. two-level factors. Journal of Statistical Planning and Inference, Schwabe, R., Graßhoff, U., Großmann, H., & Holling, H. (2003) 142, 2395–2401. https://doi.org/10.1016/j.jspi.2012.02.033 Optimal 2K paired comparison designs for partial profiles. In: Hoyos, D. (2010). The state of the art of environmental valuation PROBASTAT 2002, Proceedings of the Fourth International with discrete choice experiments. Ecological Economics, 69, Conference on Mathematical Statistics, Smolenice, 1595–1603. https://doi.org/10.1016/j.ecolecon.2010.04.011 PROBASTAT 2002, Proceedings of the Fourth Jaynes, J., Wong, W. K., & Xu, H. (2016). Using blocked International Conference on Mathematical Statistics, fractional factorial designs to construct discrete choice Smolenice. Slovakia, 26:79–86 experiments for healthcare studies. Statistics in Medicine, Shah, K. K., Tsuchiya, A., & Wailoo, A. J. (2015). Valuing 35(15), 2543–2560. https://doi.org/10.1002/sim.6882 health at the end of life: A stated preference discrete choice 14 E. NYARKO experiment. Social Science & Medicine, 124, 48–56. https:// Sudhir, K., Priester, J., Shum, M., Atkin, D., Foster, A., doi.org/10.1016/j.socscimed.2014.11.022 Iyer, G., Jin, G., Keniston, D., Kitayama, S., Mobarak, M., Soetevent, A. R., & Kooreman, P. (2007). A discrete-choice Qian, Y., Tewari, I., & Wood, W. (2015). Research oppor- model with social interactions: With an application to tunities in emerging markets: An inter-disciplinary per- high school teen behavior. Journal of Applied spective from marketing, economics, and psychology. Econometrics, 22, 599–624. https://doi.org/10.1002/jae. Customer Needs and Solutions, 2(4), 264–276. https://doi. 924 org/10.1007/s40547-015-0044-1