Generic-reference and generic-generic bioequivalence of forty-two, randomly-selected, on-market generic products of fourteen immediate-release oral drugs

The extents of generic-reference and generic-generic average bioequivalence and intra-subject variation of on-market drug products have not been prospectively studied on a large scale. We assessed bioequivalence of 42 generic products of 14 immediate-release oral drugs with the highest number of generic products on the Saudi market. We conducted 14 four-sequence, randomized, crossover studies on the reference and three randomly-selected generic products of amlodipine, amoxicillin, atenolol, cephalexin, ciprofloxacin, clarithromycin, diclofenac, ibuprofen, fluconazole, metformin, metronidazole, paracetamol, omeprazole, and ranitidine. Geometric mean ratios of maximum concentration (Cmax) and area-under-the-concentration-time-curve, to last measured concentration (AUCT), extrapolated to infinity (AUCI), or truncated to Cmax time of reference product (AUCReftmax) were calculated using non-compartmental method and their 90% confidence intervals (CI) were compared to the 80.00%–125.00% bioequivalence range. Percentages of individual ratios falling outside the ±25% range were also determined. Mean (SD) age and body-mass-index of 700 healthy volunteers (28–80/study) were 32.2 (6.2) years and 24.4 (3.2) kg/m2, respectively. In 42 generic-reference comparisons, 100% of AUCT and AUCI CIs showed bioequivalence, 9.5% of Cmax CIs barely failed to show bioequivalence, and 66.7% of AUCReftmax CIs failed to show bioequivalence/showed bioinequivalence. Adjusting for 6 comparisons, 2.4% of AUCT and AUCI CIs and 21.4% of Cmax CIs failed to show bioequivalence. In 42 generic-generic comparisons, 2.4% of AUCT, AUCI, and Cmax CIs failed to show bioequivalence, and 66.7% of AUCReftmax CIs failed to show bioequivalence/showed bioinequivalence. Adjusting for 6 comparisons, 2.4% of AUCT and AUCI CIs and 14.3% of Cmax CIs failed to show bioequivalence. Average geometric mean ratio deviation from 100% was ≤3.2 and ≤5.4 percentage points for AUCI and Cmax, respectively, in both generic-reference and generic-generic comparisons. Individual generic/reference and generic/generic ratios, respectively, were within the ±25% range in >75% of individuals in 79% and 71% of the 14 drugs for AUCT and 36% and 29% for Cmax. On-market generic drug products continue to be reference-bioequivalent and are bioequivalent to each other based on AUCT, AUCI, and Cmax but not AUCReftmax. Average deviation of geometric mean ratios and intra-subject variations are similar between reference-generic and generic-generic comparisons. ClinicalTrials.gov identifier: NCT01344070 (registered April 3, 2011).


Background
One of the causes of economic inefficiency in healthcare is underuse of generic drug products [1], which is due, in part, to mistrust by healthcare professionals [2] and patients [3] and may be related to information availability [4], educational level [3], and healthcare system maturity [2,5,6].
An application for marketing approval of a generic drug product must provide evidence of its bioequivalence (BE) to a reference product that was approved based on clinical trials [7][8][9]. Although there are some differences among regulatory agencies worldwide [7][8][9], for immediate-release drugs, average bioequivalence (BE) testing is commonly performed in a single-dose, crossover study on healthy volunteers under fasting condition; with measurement of parent drug blood concentration, non-compartmental analysis of logarithmically transformed area-under-the-concentration-time curve (AUC) and maximum concentration (C max ) data, and computation of the 90% confidence interval (CI) on the test/reference geometric mean ratio, which should generally fall within the 80-125% BE range [10,11].
Under current regulations, BE studies among onmarket, reference-bioequivalent, generic products are not required, which raises the theoretical concern that a generic product at one end of the BE range might not be equivalent to another at the other end [23][24][25]. Few studies have addressed the issue; using retrospective analysis of reference-normalized data [26][27][28], simulation [29,30], or a prospective but restricted approach [31].
One size-fits-all BE approach may not adequately take intra-subject variability and therapeutic windows into account [32][33][34]. Intra-subject variability can be due to intra-drug variability (physiological metabolic variability), intra-product variability (unit to unit or batch to batch), or subject-by-product interaction. Generic intra-product variability and subject-by-product interaction are especially important for narrow therapeutic index (NTI) drugs, for which the 75/75 rule (75% of individual ratios are within ±25%), among other methods of analysis, have been proposed [10,35]. A simulation study was assuring [25] and few studies specific to antiepileptic medications [17,28,31] provided further support of the applicability of current BE standards to NTI drugs and led to revision of the American Epilepsy Society's guidelines concerning reference-to-generic and generic-to-generic switching [36]. However, there are still concerns that the results may not apply to countries with less stringent control over pharmaceuticals' quality [37].
In Saudi Arabia, the Saudi FDA requires demonstration of BE (applying the 80.00-125.00% BE limits on C max and AUC 90% CIs) before registering generic drug products, registered products are listed in the Saudi National Formulary, generic substitution for none-NTI drugs by pharmacists is permissive with patient's consent, and generic prescribing is encouraged [38]. Although the Saudi FDA has a policy to reexamine the products for which it receives complaints, it does not systemically assess the BE of on-market generic products. A 2015 study on a random sample of 178 physicians in 2 hospitals in the Riyadh showed that although 52% supported substitution by local generic products, only 22% believed that Saudi FDA-approved, local generic products are therapeutically equivalent to reference products [39].
Given the tremendous cost-saving and potential improvement in healthcare accessibility provided by generic drug products, the serious clinical implications of prescribing products with unacceptable bioavailability or switching between products that are not bioequivalent, the need to alleviate patients and healthcare professionals mistrust, and the paucity of empirical data world-wide, we set the present study as a field test of the current BE standards. Our main aim was to determine the extent of BE between on-market generic and reference products and among referencebioequivalent generic products. We also examined the percentages of individual, generic/reference and generic/ generic, pharmacokinetic parameters ratios that are outside the ±25% range.

Design
We identified the 15 oral, immediate-release, noncombinational drugs with the highest number of generic products on the Saudi National Formulary. We studied 14 out of the 15 drugs because the reference (R) product of one of them (enalapril) was not available on the Saudi market. On each drug, we conducted four-product, foursequence, four-period, sequence-randomized, crossover BE study using the R product and 3 randomly-selected generic products (Ga, Gb, and Gc). The four sequences, namely, Ga-Gb-Gc-R, Gb-R-Ga-Gc, Gc-Ga-R-Gb, and R-Gc-Gb-Ga, were designed so that every product appears the same number of time within each period and each sequence, and every product follows every other product the same number of times. Washout periods and blood sampling frames were drug-specific (Table 1) and extended to about 7 and 5 drug plasma half-lives, respectively.

Participants
We enrolled healthy, non-pregnant adults (age 18-60 years) with a body mass index (BMI) ≤35 kg/m 2 , who accepted to abstain from taking any medication for ≥2 weeks before, and during the study, and from smoking, alcohol, and xanthene-containing beverages or food for ≥48 h before, and during each of the four study periods. Volunteers were screened by medical history, physical exam, and laboratory tests that included complete blood count, renal profile, and liver profile. Subjects with history of hypersensitivity to the drug to be tested, recent acute illness, or clinicallyimportant laboratory tests' abnormality were excluded. For menstruating women, the study was conducted 5 to 19 days after last menstrual period and after obtaining a negative urine pregnancy test.
The The study could not distinguish product failure from failure to take the drug b All adverse events were minor and resolved spontaneously. HPLC High performance liquid chromatography, LC-MS Liquid chromatography-mass spectrometry, BP Blood pressure. Flu-like, influenza-like consent at enrolment and was compensated based on the Wage-Payment model [40] in a prorated manner.

Procedures and interventions
Reference and generic drug products were purchased from retail pharmacies in Riyadh, Saudi Arabia. After fasting for 10 h, drug products were administered with 240 ml of water at room temperature. Fasting from food and beverages continued for 4 h post-dosing. However, volunteers were allowed 120 ml water every hour, except for 1 h before and 1 h after drug administration. Standardized breakfast and standardized dinner were given 4 and 10 h after drug administration. Meal plans were identical in the four study periods. Volunteers remained ambulatory or seated upright (unless deemed medically necessary) for 4 h after drug administration. Strenuous physical activity was not permitted during study periods.
During each study period, in addition to a baseline blood sample, 17 blood samples were drawn (Additional file 1). Sampling schedules were drug specific and were designed to collect adequate number of samples before and around the expected C max and across 5 half-lives of the drug. Blood samples were collected in vacutainer tubes and centrifuged for 10 min at room temperature within 15 min of collection. Plasma samples were harvested in clean polypropylene tubes and placed immediately at -80 o C until analysed.
Compliance with study protocol was checked before drug administration in each study period. Volunteers were under continuous observation regarding occurrence of adverse events and compliance with study protocol during the first day of each period. In addition, they were asked about experiencing adverse events at the time of last blood collection of each period and at the beginning of subsequent periods.
Drug concentrations were blindly measured by inhouse, locally-validated, reversed-phase high performance liquid chromatography (HPLC) [41][42][43][44][45][46][47][48][49][50][51][52] or liquid chromatography-mass spectrometry (LC-MS) [53,54]. Lower limits of quantification are listed in Table 1. Intra-assay coefficient of variation (standard deviation/ mean * 100) and bias (measured concentration/nominal concentration * 100) were ≤3.1-14.4 and ≤5.0-17.0, respectively. A typical assay run included a series of 10 calibrators and several sets of four quality control samples (1 and 3 times lower quantification limit and 0.5 and 0.8-0.9 upper quantification limit). Samples from the four periods for each volunteer were analyzed in the same run. Samples with drug concentration greater than the upper quantification limit were reassayed after dilution. Samples with drug concentration below the lower quantification limit were assigned zero concentration. Drug concentrations of missing samples were assigned the average concentration of the two flanking samples in the same period.

Random sampling of generic drug products and randomization
For each of the 14 drugs, all of the Saudi formulary-listed generic products were assigned sequential numbers, the numbers were arranged randomly (by MMH) using an online random number generator [55], and the three generic products corresponding to the first three randomly-arranged numbers were selected and labeled Ga, Gb, and Gc, respectively.
For each of the 14 studies, blocked (block size = 4) randomization sequences were generated (by MMH) using an online program [55]. Randomization sequences were concealed from recruiting study coordinators and from potential participants.

Sample size
Sample size for each study was estimated using an online program [56]; assuming an AUC I and C max ratio of generic to reference product of 1.10, a power of 0.9, a left equivalence limit of 0.80, a right equivalence limit of 1.25, and 2 one-sided type I error of 0.05, Bonferroniadjusted for 6 comparisons (i.e., α = 0.0083). Sample size was rounded and inflated by 3-8 subjects to allow for potential withdrawals/dropouts. Intra-subject coefficient of variation (CV) was estimated from published studies as 50% of reported total CV (Additional file 2).

Outcome measures and analysis
The following pharmacokinetic parameters were determined using standard non-compartmental methods: AUC T (area-under-the-concentration-time curve from time zero to time of last measured concentration) calculated by linear trapezoidal method, AUC I (area-under-the-concentrationtime curve from time 0 to infinity) calculated as AUC T plus the ratio of last measured concentration to elimination rate constant, AUC T / AUC I , C max (maximum concentration) determined directly from the observed data, T max (first time of maximum concentration) determined directly from the observed data, λ (apparent first-order elimination rate constant) calculated by linear least-squares regression analysis from the last 4-8 quantifiable concentrations of a plot of natural log-transformed concentration versus time curve, t ½ (terminal elimination half-life) calculated as ln 2/ λ, AUC 72 (area-under-the-concentration-time curve truncated to 72 h) calculated by linear trapezoidal method, and AUC Reftmax (area-under-the-concentration-time curve to T max of reference product, calculated for each subject) calculated by linear trapezoidal method. When λ was not calculable in a given study period, the average of λs in other periods of the same volunteer was used to calculate AUC I for that period. AUC Reftmax was not calculated when data for the reference product were missing. Each generic AUC Reftmax with zero value was assigned 0.001 in order to perform log-transformation. Pharmacokinetic and statistical analyses included all evaluable data of all volunteers.
Primary outcome measures were C max , AUC T , and AUC I . Secondary outcome measures were T max , AUC Reftmax, and AUC 72 . The four products of each drug were compared by analysis of variance (ANOVA). The ANOVA model included, product, period, sequence, and subjects nested in sequence. Mean square residual (MSR) was used to test significance of period and product effects. Subjects nested in sequence mean square was used to test significance of sequence effect. For each pharmacokinetic parameter (except T max ), six pairwise (Ga-R, Gb-R, Gc-R, Ga-Gb, Gb-Gc, and Ga-Gc) 90% CIs on the difference between means of log-transformed values (i.e., geometric mean ratio) were determined using MSR without and with Bonferroni adjustment for 3 or 6 comparisons, and the antilogs of the 90% CI limits were compared to the BE limits of 80.00% and 125.00%. The null hypothesis (lack of bioequivalence) was rejected if the 90% CI was completely within 80.00% to 125.00%. If the null hypothesis was not rejected, the analysis would indicate either failure to show bioequivalence (the 90% CI crosses the BE limits) or bioinequivalence (the 90% CI is completely outside the BE limits). to The following were also calculated: percentage of generic products that are not bioequivalent to their reference product or not bioequivalent to each other based on C max , AUC T , AUC I , or AUC Reftmax , mean (SD) deviation of AUC T , AUC I , and C max genericreference and generic-generic point estimates from 100% and percentages of the deviations that were <6, <10, or >13 percentage points, percentage of individual C max , AUC T , AUC I , AUC 72 , T max , and AUC Reftmax generic/reference and generic/generic ratios that are 75% or 125%, and percentage of drugs that failed to fulfil the 75/75 rule (i.e.,75% of individual ratios are within ±25%) for each of the pharmacokinetic parameters. Pharmacokinetic and statistical analyses were performed (by MMH) on a personal computer using Microsoft Excel (Version 2010) with add-ins (PK Functions for Microsoft Excel, JI Usansky, A Desai, and D Tang-liu, Department of pharmacokinetics and Drug Metabolism, Allergan Irvine, CA, USA) and IBM SPSS Statistics version 21 software, respectively.

Results
The 14 immediate-release, non-combinational, oral drugs with the highest number of generic products on the Saudi National Formulary that were assessed were, in descending order, ciprofloxacin (18 generic products), ranitidine, amoxicillin, paracetamol, atenolol, cephalexin, ibuprofen, diclofenac, metformin, omeprazole, metronidazole, clarithromycin, amlodipine, and fluconazole (7 generic products). Commercial name, manufacturer name, formulation, strength, lot/ batch number, manufacture date, and expiry date for the reference and the 3 randomly-selected generic products as well as the number of listed generic products are presented in Additional file 3. About 52% of the 42 generic products were manufactured in Saudi Arabia, 14% in other Gulf States, 31% in Arabic non-Gulf States, and 2% in Portugal.
Seven hundred healthy volunteers participated in 14, four-product, four-sequence, four-period, sequencerandomized, crossover, BE studies. As shown in Table 1, the number of volunteers per study ranged from 28 to 80. The volunteers were 100% males for all but 3 studies which had 3-6% females. Mean (SD) age ranged from 30.5 (5.0) to 36.9 (8.7) years and mean BMI ranged from 23.0 (2.3) to 26.1 (3.7) kg/m 2 per study (grand mean age and BMI 32.2 (6.2) years and 24.4 (3.2) kg/m 2 , respectively). Withdrawal from ≥ one period ranged from 0% to 19% per drug, with a total of 145 missed periods (out of 2800). Withdrawal reasons were mostly personal but also included inadequate venous access, skin rash, vomiting, high blood pressure, and influenza-like symptoms, as well as incompliance (Table 1). Adverse events occurred in 0% (paracetamol) to 7% (fluconazole and metronidazole) of volunteers (Table 1); all were minor and resolved spontaneously.
Baseline drug concentration was not detectable in any period for any of the 14 drugs, indicating adequate wash-out periods. There were 12 missed blood samples (2 for clarithromycin, 5 for fluconazole, and 5 for ranitidine) out of the 47,790 scheduled samples (excluding withdrawals); these samples were assigned the average concentration of the two flanking samples of the same volunteer in the same period. In all samples of one volunteer, there was a plasma peak that interfered with the diclofenac assay; this volunteer was excluded from further analysis. In four volunteers, there was no measurable drug concentration in any sample from one study period only (amlodipine, R, 3rd period; ibuprofen, R, 2nd period; metronidazole, Gb, 1st period; and paracetamol, Gb, 2nd period). The unmeasurable concentrations could be due to product failure as the drugs were administered by one of the investigators and the volunteers denied incompliance when confronted; however, incompliance cannot be ruled out. Mean concentrationtime and log-concentration-time curves of the reference and the three generic products of each of the 14 drugs are presented in Additional files 4 and 5, respectively. We were not able to calculated λ in a total of 27 (1%) out of the 2647 pharmacokinetic analyses (clarithromycin: (1) Ga, (3) Gb, and (1) Gc; diclofenac: (4) Ga, (4) Gb, (3) Gc, and (7) R; omeprazole: (1) Gb, (2) Gc, and (1) R). Average of λs in other periods of the same volunteer was used to calculate AUC I for these 27 analyses.
No outlier values for any of the pharmacokinetic parameters were identified or removed from analysis. AUC T , AUC I , C max , T max , λ, t 1/2 , C max /AUC I , AUC T /AUC I , AUC Reftmax , and AUC 72 of the reference and the three randomly-selected generic products of each drug are summarized in Additional file 6. AUC T /AUC I ranged from 90% (ciprofloxacin) to 98% (clarithromycin), indicating adequate sampling frames.
MSR from ANOVA analysis and calculated intrasubject CV for AUC T , AUC I , and C max of each drug are presented in Table 2. Significant product, period, and sequence effects on AUC T , AUC I , and C max of the 14 drugs are summarized in Additional file 7. MSR and intra-subject CV for AUC Reftmax and AUC 72 are presented in Additional files 8 and 9, respectively.
Average bioequivalence of 3 on-market generic products to the reference product of 14 drugs Table 2 summarizes the results of the 42 predetermined BE analyses comparing three randomly-selected generic products to the corresponding reference product of each of the 14 drugs. The results are also depicted in Fig. 1. None of the AUC T or AUC I 90% CIs failed to show bioequivalence and 9.5% of C max 90% CIs only barely failed to show bioequivalence. When analyses were adjusted for 3 comparisons, 2.4% of AUC T 90% CIs, 0% of AUC I 90% CIs, and 11.9% of C max 90% CIs failed to show bioequivalence, and none showed bioinequivalence. When analyses were adjusted for 6 comparisons, 2.4% of AUC T 90% CIs (clarithromycin Gc vs. R), 2.4% of AUC I 90% CIs (clarithromycin Gc vs. R), and 21.4% of C max 90% CIs (clarithromycin Ga and Gc vs. R; diclofenac Ga, Gb, and Gc vs. R; ibuprofen Gb and Gc vs. R; omeprazole Gb and Gc vs. R) failed to show bioequivalence, and none showed bioinequivalence.
Mean absolute (SD) deviation of point estimates from 100% in the 42 comparisons was 3.2 (1.8), 3.2 (1.4), and 5.4 (3.3) percentage points for AUC T , AUC I , and C max , respectively. Further, the deviation was 10 percentage points in 95.2%, 95.2%, and 81.0% of the AUC T , AUC I , and C max comparisons, respectively. Furthermore, 0 % of the AUC T and AUC I and 9.5% of the C max deviations were >13 percentage points and 78.6%, 81.0%, and 50.0%, respectively, were <6 percentage points. Figure 2 (a) depicts BE analysis of AUC Reftmax between the three generic products and the corresponding reference product of each of the 14 drugs. The data are also summarized in Additional file 8. Twenty two (52.4%) of the 90% CIs failed to show bioequivalence. In addition, 6 (14.3%) showed bioinequivalence. Figure 2 (b) depicts BE analysis of AUC 72 between the three generic products and the corresponding reference product of the two drugs with long half-life (amlodipine and fluconazole). BE was demonstrated by all of the six 90% CIs. The data are also summarized in Additional file 9.
Individual pharmacokinetic parameter ratios of 3 onmarket generic products to the reference product of 14 drugs There were 1950 individual generic-reference comparisons. The percentages of individual AUC T, AUC I , and C max, ratios that were outside the ±25% range are presented in Fig. 3. On average, 16% of the AUC T ratios (ranging from 2% for cephalexin to 35% for atenolol and clarithromycin), 15% of the AUC I ratios (ranging from 2% for cephalexin to 34% for clarithromycin), and 32% of C max ratios (ranging from 8% for metronidazole to 57% for diclofenac), were outside the ±25% range. Further, individual AUC T , AUC I , and C max, ratios were within the ±25% range in 75% of individuals (i.e., fulfilled the 75/75 rule) for 79%, 79%, and 36% of the 14 drugs, respectively.
Out of 161 and 76 AUC 72 individual ratios for amlodipine and fluconazole, 16% and 1%, respectively, were outside the ±25% range (compared to 18% and 3%, respectively, for AUC T ). Figure 4 depicts the percentages of individual generic/ reference T max and AUC Reftmax ratios that were outside the ±25% range. On average, 60% of the T max ratios (ranging from 43% for amoxicillin to 72% for ibuprofen) and 58% of the AUC Reftmax ratios (ranging from 27% for metformin to 89% for omeprazole) were outside the ±25% range. Individual T max and AUC Reftmax ratios were within the ±25% range in 75% of individuals for none of the 14 drugs, respectively.
Average bioequivalence among 3 on-market generic products of 14 drugs Table 2 also summarizes the results of the 42 predetermined BE analyses among the three randomly-selected generic products of each of the 14 drugs. The results are also depicted in Fig. 5. Only one (2.4%) of each of the AUC T , AUC I , and C max 90% CIs failed to show bioequivalence. When analyses were adjusted for 3 comparisons, 2.4% of AUC T and AUC I 90% CIs and 9.5% of C max 90% CIs failed to show bioequivalence, and none showed bioinequivalence. When analyses were adjusted for 6 comparisons, 2.4% of AUC T and AUC I (clarithromycin Gb vs. Gc) and 14.3% of C max 90% CIs (cephalexin Ga vs. Gb and Gb vs. Gc; clarithromycin Gb vs. Gc and Ga vs. Gc; ibuprofen Gb vs. Gc and Ga vs. Gc) failed to show bioequivalence, and none showed bioinequivalence.
Mean absolute (SD) deviation of point estimates from 100% in the 42 comparisons was 2.5 (2.3), 2.6 (2.2), and 3.3 (3.1) percentage points for AUC T , AUC I , and C max , respectively. Further, the deviation was <10 percentage points in 95.2%, 95.2%, and 88.1% of the AUC T , AUC I , and C max comparisons, respectively. Furthermore, only 2.4% of the AUC T and AUC I and 7.1% of the C max  deviations were >13 percentage points and 81.0%, 81.0%, and 59.5%, respectively, were <6 percentage points. Figure 6 (a) depicts BE analysis of AUC Reftmax among the three generic products of each of the 14 drugs. The data are also summarized in Additional file 8. Twenty three (54.8%) of the 90% CIs failed to show bioequivalence. In addition, 5 (11.9%) showed bioinequivalence. Figure 6 (b) depicts BE analysis of AUC 72 among the three generic products of the two drugs with long halflife. BE was demonstrated by all of the six 90% CIs. The data are also summarized in Additional file 9.

Individual pharmacokinetic parameter ratios among 3 onmarket generic products of 14 drugs
There were 1952 individual generic-generic comparisons. The percentages of individual AUC T , AUC I , and C max, ratios that were outside the ±25% range are presented in Fig. 7. On average, 17% of the AUC T ratios (ranging from 1% for metronidazole and fluconazole to 40% for clarithromycin), 16% of the AUC I ratios (ranging from 1% for metronidazole and fluconazole to 38% for clarithromycin), and 32% of the C max ratios (ranging from 5% for fluconazole to 59% for diclofenac) were outside the ±25% range. Further, individual AUC T , AUC I , and C max ratios were within the ±25% range in ˃75% of individuals for 71%, 71%, and 29% of the 14 drugs, respectively, Out of 161 and 76 AUC 72 individual ratios for amlodipine and fluconazole, 19% and 1%, respectively, were outside the ±25% range (compared to 25% and 1%, respectively, for AUC T ). Figure 8 depicts the percentages of individual generic/ generic T max and AUC Reftmax ratios that were outside the ±25% range. On average, 58% of the T max ratios (ranging from 42% for amlodipine to 73% for fluconazole) and 52% of the AUC Reftmax ratios (ranging from 18% for fluconazole to 82% for omeprazole) were outside the ±25% range. Individual T max and AUC Reftmax ratios were within the ±25% range in >75% of individuals for 0% and 7% of the 14 drugs, respectively.

Discussion
We assessed the adequacy of the commonly-used BE standards and of their application in a developing country through determining BE extent between onmarket generic and reference drug products and among reference-bioequivalent generic drug products. We studied 42 generic products of 14 immediate-release, non-combinational, oral drugs with the highest number of generic products on the Saudi market. We conducted a four-product, four-period, four-sequence, sequencerandomized, crossover BE study with a planned power of 0.9 on a reference and three randomly-selected generic products of each of the 14 drugs. For each drug, we computed six pairwise 90% CIs on geometric mean ratios of AUC T , AUC I , C max , AUC Reftmax, and AUC 72 without and with adjustment for multiple comparisons and determined percentages of individual untransformed ratios that fell outside the ±25%. We found that: 1) Onmarket generic drug products continue to be referencebioequivalent. 2) Reference-bioequivalent generic products are bioequivalent to each other. 3) Reference-generic AUC T is the area-under-the-concentration-time curve to last measured concentration. AUC I is AUC extrapolated to infinity. C max is maximum concentration. Data represent geometric mean ratios and unadjusted 90% confidence intervals. The number of subjects analyzed in each comparison is presented between parentheses in the first column. MSR is mean square residual from analysis of variance (ANOVA). CV is intra-subject coefficient of variation calculated as 100 x (exp(MSR)-1) 0.5 . Confidence intervals that cross the 80.00%-125.00% bioequivalence limits are bolded and generic-generic average deviations are small and similar. 4) Reference-generic and generic-generic C max intra-subject variations are large but similar. 5) Two thirds of generic-reference and generic-generic AUC Reftmax comparisons failed to show average bioequivalence/ showed bioinequivalence. The number of generic products for an off-patent drug is usually related to its market size. Therefore, it is reasonable to assume that the 14 drugs that we studied are among the commonly prescribed drugs in Saudi Arabia. They happened to include drugs for which rapid onset of action is clinically relevant (paracetamol, ibuprofen, diclofenac), drugs that are used in chronically and for which the concept of switchability is relevant (metformin, amlodipine), drugs with long half-life (fluconazole, amlodipine), and highly variable drugs (clarithromycin, diclofenac), but not NTI drugs. Almost all of the generic products were manufactured in Saudi Arabia or in a Middle Eastern state.
Marketed generic products of immediate-release, noncomputational, oral drugs continue to be bioequivalent to their corresponding reference products A generic drug product is commonly approved for continued marketing based on a single pre-marketing study demonstrating BE to its reference product; retesting of BE post-marketing is not routinely required. Our results confirm the validity of such practice. Using the 80.00-125.00% BE range, we found that 100% of the AUC T and AUC I generic-reference 90% CIs showed BE and only 9.5% of the C max 90% CIs barely failed to show BE. Even after adjusting for 6 comparisons, only 2.4% of the AUC T and AUC I 90% CIs and 21.4% of the C max 90% CIs failed to show BE. Our results are in line with some [17,22] but not all [15,16] published studies. a b c Fig. 1 Average bioequivalence of randomly-selected generic products to the reference product of 14 immediate-release, non-combinational, oral drugs. Each reference product (R) was compared to 3 generic products (Ga, Gb, Gc). Data represent generic/reference geometric mean ratios and unadjusted 90% confidence intervals. The shaded area indicates the area of bioequivalence (80.00%-125.00%). a Evaluation of area-underthe-concentration-time curve to last measured concentration (AUC T ). b Evaluation of area-under-the-concentration-time curve extrapolated to infinity (AUC I ). c Evaluation of maximum concentration (C max ) Previous studies evaluated generic products on other national markets, examined only one [17] or two [16,22] generic products of a single drug, or were not performed in vivo [15].
The outcome of a crossover BE study is affected by its sample size and intra-subject variability [57]. We estimated intra-subject CVs from published studies and planned each of the 14 studies to have a power of 0.9. It is of note that for the 4 drugs that failed to show BE in some of the comparisons (clarithromycin, diclofenac, ibuprofen, and omeprazole), current study intra-subject CVs were larger than estimated (Additional file 2). Intra-subject variability can be related to inter-product variability; however, it can be also attributed to the drug substance itself (being readily affected by intra-subject physiological variability), intraproduct variability, analytical variability, or unexplained random variability [57]. In fact, in a separate study [58] that compared the reference ibuprofen product used in this study to itself, using the same settings and a larger sample size, the C max 90% CI also failed to show BE. This suggests that at least some of the failures to show BE in the current study may not be due to real genericreference (inter-product) differences.
We found that the mean deviation of the generic/reference ratio from 100% was 3.2%, 3.2%, and 5.4% for AUC T , AUC I , and C max , respectively, and that the deviation was <10 percentage points in 95.2%, 95.2%, and 81.0% of the 42 comparisons. Similarly, the US FDA found a mean deviation of 3.47% for AUC T and 4.29% for C max in one retrospective study [59] and 3.56% for AUC T and 4.35% for C max in another [60], and that in about 98% of the studies, the AUC T difference was <10% [60]. Further, a reanalysis of 141 US FDA-approved antiepileptic generic products found that generic and reference AUC T and C max differed by <15% in 99% and 89% of BE studies, respectively [28]. Consistent with these BE findings, several metaanalysis and reviews showed that there is no evidence that cardiovascular [18,19], antiepileptic [20], or immunosuppressive [21] reference drug products are superior to their generic counterparts in terms of efficacy or side effects.
Reference-bioequivalent generic drug products continue to be underused world-wide, mainly due to mistrust by healthcare professionals [2] and patients [3], in a way that may be dependent on maturity of the country's healthcare system [2,5,6]. The misbelief that generic medicines are counterfeits and the placebo effect of packaging and price Fig. 2 Average bioequivalence of randomly-selected generic products to the reference product of 14 immediate-release, non-combinational, oral drugs. Each reference product (R) was compared to 3 generic products (Ga, Gb, Gc). Data represent generic/reference geometric mean ratios and unadjusted 90% confidence intervals. The shaded area indicates the area of bioequivalence (80.00%-125.00%). a Evaluation of area-underthe-concentration-time curve to time of maximum concentration of reference product, calculated for each subject (AUC Reftmax ). b Evaluation of areaunder-the-concentration-time curve truncated to 72 h (AUC 72 ). Only 2 drugs (amlodipine and fluconazole) in this study have terminal half-life >72 h differential are important to consider [61]. Further, prescribing a generic product by its brand name rather than its non-proprietary name (generic prescribing) may better convey the impression of individuality and improve patients' acceptance [62,63]. Importantly, information availability to healthcare professionals and patients has been identified as a facilitator of generic products uptake [4,39]. Our results provide strong supporting evidence of the post-marketing quality of generic products and of the adequacy of the current BE standards.
Marketed, reference-bioequivalent, generic products of immediate-release, non-combinational, oral drugs are bioequivalent to each other Commonly, there are several same-market drug products that are linked by a chain of reference; theoretical concerns have been raised that reference-bioequivalent generic products may not be bioequivalent to each other if their BE point estimates were on the opposite sides within the BE range [23,24]. Simulation studies predicted that two reference-bioequivalent generic products are likely to be equivalent to each other only under relatively restricted conditions [29,30]. However, using reference-normalized data to indirectly estimate 90% CIs, analysis of 19 BE studies on 2 anti-epileptic drugs showed generic-generic BE in almost all cases [26] and analysis of 120 BE studies on three immunosuppressants as well as six selected drugs showed BE in 90% of AUC T and 87% of C max comparisons with mean absolute deviation from 100% of 4.5% for AUC T and 5.1% for C max [27]. Further, a similar analysis of US FDA-approved antiepileptic generic products found that AUC T and C max differed by >15% in 17% and 39% of simulated generic-generic switches, respectively [28]. Nevertheless, there is little direct empirical evidence regarding the extent of BE among reference-bioequivalent generic products; two amoxicillin generic products did not show BE [16], whereas two metformin generic products [22] and the two most disparate generic lamotrigine products [31] did. In our prospective study of 42 direct generic-generic BE comparisons, only one (2.4%) comparison failed to show BE because of C max and one because of AUC T and AUC I . After adjusting for 6 comparisons, the percentages were 2.4% and 14.3%, respectively. Further, mean deviation of generic/generic ratio from 100% was only 2.5%, 2.6%, and 3.3% for AUC T , AUC I , and C max , respectively, and the deviation was <10 percentage points in 95.2%, 95.2%, and 88.1% of the 42 comparisons. Our results provide strong empirical evidence that it is very unlikely for two reference-bioequivalent generic products not to be bioequivalent to each other. Interestingly, in our study, mean deviation of generic/reference ratios from 100% was in the 6-13 percentage points range in 21.4%, 19%, and 40.5% of the AUC T and, AUC I , and C max comparisons, respectively. This suggests that, contrary to the result of previous simulation study [29], even when the bioavailability difference between generic and reference products is in the 6-13 percentage points range, reference-bioequivalent generic products are still likely to be bioequivalent.
Theoretically, the change in drug exposure resulting from generic-generic substitution might be expected to be more pronounced than the change resulting from generic-reference substitution [23,24]. However, our results indicate that the two changes in exposure are similar. Mean absolute deviation of point estimates in percentage points was 3.2 vs. 2.5 for AUC T , 3.2 vs. 2.6 for AUC I , and 5.4 vs. 3.3 for C max in the genericreference and generic-generic comparisons, respectively. Further, the deviations were <10 percentage points in similar proportions of the two types of comparisons.

Generic-reference and generic-generic intra-subject variability of bioequivalent drug products
Since average BE focuses on mean difference rather than difference between variances or subject-by-product interaction, it is possible that a patient on a referencebioequivalent but low-quality generic product may be sometimes overdosed and sometimes underdosed and that a patient using two bioequivalent products may have the highest drug exposure with one product and the lowest with another [64]. Such possibilities may be of particular concern when switching patients form one NTI drug product to another [24] and are usually reflected in individual ratios of the pharmacokinetic parameters. Few published studies have addressed BE at the individual level [17,24,25]. Despite having 90% CIs within the 80-125% limits, 18% and 38% of individual cyclosporine generic/reference AUC and C max ratios, respectively, were <0.80 [24] and 0% of individual lamotrigine generic/reference AUC and C max ratios and 3% and 18% of same-product, generic/generic AUC and C max ratios, respectively, were outside the ±25% range [17]. A simulation study (assuming 20% inter-subject variability and 10% intra-subject variability) predicted that when mean generic product's AUC is 80% to 123.5% of reference product's AUC, 3-4.6% and 9-12% of individual generic/reference and generic/generic AUC ratios, respectively, would fall outside the 0.67-1.5 range [25].
We found that 16% and 17% of individual generic/reference and generic/generic ratios, respectively, were outside the ±25% range in for AUC T , 15% and 16% for AUC T , and 32% and 32% for C max . Further, individual generic/reference and generic/generic AUC T , AUC I , and C max ratios fulfilled the 75/75 rule for 79% and 71%, 79% and 71%, and 36% and 29% of the 14 drugs, respectively. Based on a relatively large number of drug products, our results document the extent of intra-subject variability that would be expected despite fulfilment of average BE criteria and strongly suggest that the extents of generic-generic switchability and generic-reference switchability are similar.
It is not clear how much of the observed intra-subject variability is due to inter-product rather than intra-product variability. In the simulation study, 11.1% of the reference/ reference AUC ratios were predicted to fall outside the 0.8-1.25 range [25]. Further, 3% and 9% of individual lamotrigine reference/reference AUC and C max ratios [17] and 23%, 30%, and 30% of individual caffeine AUC T , AUC I , and C max ratios [65], respectively, were outside the ±25% range. Furthermore, when the cephalexin, ibuprofen, and paracetamol reference products used in this study were compared to themselves; respectively, 2%, 17%, and 2% of the individual ratios were outside the ±25% range for AUC T a b c Fig. 5 Average bioequivalence among randomly-selected, reference-bioequivalent generic products of 14 immediate-release, non-combinational, oral drugs. Three generic products (Ga, Gb, Gc) were compared. Data represent generic/generic geometric mean ratios and unadjusted 90% confidence intervals. The shaded area indicates the area of bioequivalence (80.00%-125.00%). a Evaluation of area-under-the-concentration-time curve to last measured concentration (AUC T ). b Evaluation of area-under-the-concentration-time curve extrapolated to infinity (AUC I ). c Evaluation of maximum concentration (C max ) (compared to 2%, 8%, and 8% of the generic-reference ratios in the current study), 4%, 3%, and 2% for AUC I , (compared to 2%, 8%, and 9% of the generic-reference ratios in the current study), and 25%, 33%, and 45% for C max , (compared to 39%, 22%, and 26% of the genericreference ratios in the current study) [58]. Together, the data strongly indicate that a major part of the intra-subject variability seen in average BE studies may not be related to comparing two products but rather to factors such as study setting, drug assay, and random variations in subject's physiologic status (for example, gastric emptying, intestinal transit speed, and luminal pH).

Large variability in AUC Reftmax and T max despite average bioequivalence
When time of onset of drug effect is important because of therapeutic or toxic issues, it is recommended to perform non-parametric analysis of non-transformed T max values and/or evaluate the 90% CI of AUC truncated at reference T max median or at reference T max , calculated for each subject (AUC Reftmax ) [7,8]. Onset of effect may important for only few drugs in the current study, however, we used the data on all the 14 drugs to examine the behaviour of T max and AUC Reftmax in general.
We found that two thirds of generic-reference and generic-generic AUC Reftmax comparisons failed to show BE or showed bioinequivalence. Further, on average, 60% and 58% of generic/reference and 58% and 52% of generic/generic individual T max and AUC Reftmax ratios, respectively, were outside the ±25% range. Moreover, generic/reference and generic/generic individual T max and AUC Reftmax ratios fulfilled the 75/75 rule in only 0-7% of the 14 drugs. The results confirm that average BE testing using AUC T , AUC I , and C max is insensitive to variability in T max and AUC Reftmax and suggest that intra-subject variabilities of the two parameters are similar and do not depend on whether a generic product is compared to a reference product or to another generic product.
Some patients' bad impression of generic products may be theoretically related to their different onset of effect as compared to reference products. However, this is not likely because onset of effect is mostly related to pharmacodynamic rather than pharmacokinetic characteristics. Further, since T max values are based on C max , which is, in turn, based on a single measurement of drug concentration, T max values are also very sensitive to study setting, subject's physiological status, assay variability, and random error. In a b Fig. 6 Average bioequivalence among randomly-selected, reference-bioequivalent generic products of 14 immediate-release, non-combinational, oral drugs. Three generic products (Ga, Gb, Gc) were compared. Data represent generic/generic geometric mean ratios and unadjusted 90% confidence intervals. The shaded area indicates the area of bioequivalence (80.00%-125.00%). a Evaluation of area-under-the-concentration-time curve to time of maximum concentration of reference product, calculated for each subject (AUC Reftmax ). b Evaluation of area-under-the-concentration-time curve truncated to 72 h (AUC 72 ). Only 2 drugs (amlodipine and fluconazole) in this study have terminal half-life >72 h fact, when the cephalexin, ibuprofen, and paracetamol reference products used in this study were compared to themselves [58]; respectively, 46%, 63% and 71% of individual ratios were outside the ±25% range for T max (compared to 54%, 72% and 69% of the generic-reference ratios in the current study) and 71%, 77% and 67% for AUC Reftmax (compared to 75%, 76% and 68% of the generic-reference ratios in the current study). This strongly indicates that most of the observed generic-reference and generic-generic intrasubject variability in T max and AUC Reftmax is not due to inter-product differences and that the usefulness of T max and AUC Reftmax in BE evaluation may be very limited.
AUC 72 is as informative as AUC T Two drugs in this study have long plasma half-life (around 49 and 29 h); the half-life for the other 12 drugs was <10 h. We were able to demonstrate average BE in all genericreference and all generic-generic AUC T and AUC 72 comparisons. Further, similar percentages of generic/reference and generic/generic individual AUC T and AUC 72 ratios were outside the ±25% range. The results lend further support to using AUC 72 instead of AUC T for drugs with long plasma half-life [7][8][9].

Limitations
The interpretation of the results of this study may be limited by the following. 1) We only studied non-combinational drug products. However, BE standards for combinational and non-combinational products are the same and it can be assumed that the results apply to combinational products. 2) We only studied solid immediate-release drug products, thus our results may not apply to liquid or modified-release products. 3) Our results may not be generalizable to other solid immediate-release drugs on the Saudi market since the drugs we studied were not randomly selected. Short of more relevant statistics, the number of on-market generic products is a reasonable reflection of the extent of drug utilization. Further, the generic products in our study were a b c Fig. 7 Individual pharmacokinetic ratios among randomly-selected, reference-bioequivalent generic products of 14 immediate-release, noncombinational, oral drugs. Three generic products (Ga, Gb, Gc) were compared. Data represent percentage of individual generic/generic ratios that are <0.75 (closed bars) or >1.25 (open bars). a Evaluation of area-under-the-concentration-time curve to last measured concentration (AUC T ). b Evaluation of area-under-the-concentration-time curve extrapolated to infinity (AUC I ). c Evaluation of maximum concentration (C max ) randomly selected. Thus it would be expected that the results apply to an important portion of drug products on the Saudi market. 4) Although Saudi Arabia's BE regulations are very similar to most BE regulations worldwide, our results may not apply to similar drugs on other national markets. 5) Our study was not designed to partition intrasubject variability into its various components. Thus, it is not clear how much of the observed intra-subject variability is related to the generic products themselves (generic product quality variability or subject-by-product variability) and how much to methodological issues. 6) We observed significant (unadjusted) period and sequence effects in 6 and 2 of the 14 studies, respectively. It is likely that the apparent significance is due in large part to multiple comparisons and relatively large sample sizes, since we have also observed significant product effect in 8 of the 14 studies. The presence of period or sequence effect doesn't influence BE conclusions. Sequence effect and period effect may indicate unequal carryover, which is not likely given the length of the washout periods and the fact that baseline drug concentrations were undetectable in all periods for all 14 drugs. Sequence effect may also indicate that the groups (the 4 sequences) are different, which is also not likely because of randomization. However, it may also be due to product-by-period effect, which cannot be rolled out. Finally, period effect may indicate temporal changes, such as changes in patients' comfort level, familiarization with study, compliance, venous access, and drug stability. The latter is not likely because analysis of all drugs was performed well within each drug's pre-established stability period. 7) We have loss of follow up for one or more periods in 13 of the 14 studies, however, this resulted in negligible imbalance among the 4 sequences and negligible loss of power. 8) Finally, in retrospect, few of the 14 studies did not have adequate power to show BE for C max , however, this would strengthen the main conclusions of the study.

Conclusions
Based on studying 42 randomly-selected generic products of 14 immediate-release, non-combinational, oral drugs with the highest number of generic products on the Saudi market, we can conclude that: 1) On-market generic products continue to be reference-bioequivalent. 2) Referencebioequivalent generic products are bioequivalent to each other, despite the presence of some generic-reference deviations that are >6 percentage points. 3) Referencegeneric and generic-generic average deviations are small (on average 3-5 percentage points) and similar. 4) a b Fig. 8 Individual pharmacokinetic ratios among randomly-selected, reference-bioequivalent generic products of 14 immediate-release, noncombinational, oral drugs. Three generic products (Ga, Gb, Gc) were compared. Data represent percentage of individual generic/generic ratios that are <0.75 (closed bars) or >1.25 (open bars). a Evaluation of time of maximum concentration (T max ). b Evaluation of area-under-the-concentration-time curve to time of maximum concentration of reference product, calculated for each subject (AUC Reftmax )