eval

Prompt

You are evaluating a proposed operational change for a mid-sized pharmaceutical company. Your response will be used to compare the reasoning, accuracy, judgment, writing quality, and instruction-following of different AI models. Read the complete scenario before answering. SCENARIO A pharmaceutical company currently uses three contract research organizations, or CROs, to conduct nonclinical toxicology studies. During the previous 12 months: CRO Alpha conducted 18 studies. * Total invoiced cost: $8,460,000 * Studies completed by the original target date: 15 * Studies completed late: 3 * Major quality findings: 1 * Minor quality findings: 14 * Average sponsor oversight hours per study: 82 * Average final report delay after study completion: 37 days * Two late studies were caused primarily by sponsor-requested protocol amendments. * One late study was caused by a CRO staffing shortage. * The major quality finding involved incomplete documentation of a dosing deviation. The study data remained interpretable. CRO Beta conducted 12 studies. * Total invoiced cost: $5,040,000 * Studies completed by the original target date: 11 * Studies completed late: 1 * Major quality findings: 2 * Minor quality findings: 8 * Average sponsor oversight hours per study: 61 * Average final report delay after study completion: 24 days * The late study was delayed because the sponsor provided test article six days later than planned. * One major quality finding involved an incorrect dose formulation concentration that affected two animals before detection. * The second major quality finding involved failure to follow the protocol-specified sample-processing temperature. The effect on toxicokinetic data could not be fully determined. CRO Gamma conducted 9 studies. * Total invoiced cost: $3,420,000 * Studies completed by the original target date: 7 * Studies completed late: 2 * Major quality findings: 0 * Minor quality findings: 19 * Average sponsor oversight hours per study: 105 * Average final report delay after study completion: 53 days * One late study was caused by an equipment failure. * One late study was caused by delayed pathology peer review. * Eight of the minor quality findings involved recurring documentation errors. * Gamma recently replaced its study-management system and states that the new system will reduce documentation errors and report delays, but no completed-study data are available to verify that claim. The company expects to conduct 48 nonclinical studies next year. Management is considering consolidating most of the work with one preferred CRO. The proposed allocation is: * Preferred CRO: 32 studies * Second CRO: 12 studies * Third CRO: 4 studies Management’s stated objectives are: 1. Reduce total external study cost by at least 8%. 2. Reduce sponsor oversight workload. 3. Avoid increasing compliance, animal welfare, data integrity, or submission risk. 4. Improve schedule predictability. 5. Maintain enough CRO capacity to respond to unexpected development needs. The procurement department recommends CRO Gamma because it has the lowest average cost per study. The toxicology department recommends CRO Alpha because it has the largest operating capacity and the most experience with the company’s study types. The quality department recommends CRO Beta because it has the lowest number of total findings per study and the shortest average final report delay. Additional information: * Alpha has offered a 7% discount if awarded at least 28 studies. * Beta has offered a 4% discount if awarded at least 20 studies. * Gamma has offered a 12% discount if awarded at least 24 studies. * Alpha states it can support up to 36 studies next year without adding staff. * Beta states it can support up to 22 studies next year without adding staff. * Gamma states it can support up to 30 studies next year, but this would require hiring four additional study directors. * Assume the average study mix next year is comparable across the CROs unless the available information gives a reason not to. * Do not assume that every quality finding has equal severity. * Do not assume that a study completed after its original target date represents poor CRO performance when the delay was primarily sponsor-caused. * Do not treat promotional statements or planned process improvements as proven results. * A major quality finding should not automatically disqualify a CRO, but the nature and potential impact of the finding must be considered. * The company has not provided information about study complexity, animal species, regulatory jurisdiction, therapeutic area, historical change orders, or CRO-specific pricing by study type. YOUR TASK Prepare a decision analysis recommending how the 48 studies should be allocated among Alpha, Beta, and Gamma. Your response must contain the following sections in this exact order: 1. Executive Recommendation 2. Verified Calculations 3. Performance Assessment 4. Strongest Case for Each CRO 5. Recommended Allocation 6. Risks and Mitigations 7. Missing Information 8. Final Conclusion REQUIREMENTS Executive Recommendation Begin with a direct recommendation in no more than 150 words. State which CRO should receive the largest allocation and provide the recommended number of studies for all three CROs. Verified Calculations Calculate and show: * Average invoiced cost per study for each CRO * Cost per study after any applicable volume discount under your recommended allocation * Estimated total external cost of your recommended allocation * Estimated cost of allocating all 48 studies at the current weighted-average historical cost per study * Estimated dollar and percentage difference between those two amounts * On-time completion rate for each CRO using the original target dates * CRO-controllable on-time completion rate after excluding delays that were primarily sponsor-caused * Major findings per study * Minor findings per study * Total findings per study * Estimated annual sponsor oversight hours under your recommended allocation Show formulas or enough calculation detail for another person to verify the results. Do not apply a volume discount unless your recommended allocation meets the stated threshold. When calculating the current weighted-average historical cost per study, use the combined historical cost and combined historical study count across all three CROs. When calculating CRO-controllable on-time completion rates, exclude sponsor-caused late studies from both the numerator and denominator. Explain the treatment used. Performance Assessment Assess each CRO across: * Cost * Schedule performance * Quality and compliance risk * Sponsor oversight burden * Capacity and scalability * Reporting timeliness * Reliability of the available evidence Separate factual observations from interpretation. Do not rank the CROs solely by the total number of findings. Consider the severity, recurrence, detectability, and potential effect of the findings. Strongest Case for Each CRO Present the strongest defensible argument for making each CRO the preferred provider, even when you ultimately disagree with that choice. Recommended Allocation Recommend a specific allocation totaling exactly 48 studies. Explain: * Why the preferred CRO should receive the largest share * Why the other two CROs should remain in the portfolio * Whether any CRO should receive fewer studies than management’s proposed minimum of four * Whether the allocation should be immediate or phased * What performance conditions should be required before increasing any CRO’s allocation Do not assume all studies are interchangeable. Where appropriate, recommend that higher-risk or more complex studies be allocated differently from routine studies. Risks and Mitigations Identify at least eight material risks associated with your recommendation. For each risk, provide: * The risk * Why it matters * A practical mitigation * An early warning indicator Missing Information Identify the missing information that could materially change the recommendation. Rank the five most important missing items from highest to lowest priority. Explain how each item could change the decision. Final Conclusion End with: * Your final allocation * Your preferred CRO * Your confidence level as a percentage * The three assumptions that most limit your confidence * The specific evidence that would cause you to change the preferred CRO ANALYTICAL RULES * Do not invent information. * Do not silently resolve ambiguities. * Clearly label assumptions. * Clearly distinguish observed facts, calculations, interpretations, and recommendations. * Challenge the procurement, toxicology, and quality department recommendations rather than accepting any of them at face value. * Identify any misleading comparison created by raw totals, small sample sizes, or differences in study volume. * Consider concentration risk and the operational consequences of consolidating work. * Consider whether Gamma’s staffing plan creates execution risk. * Consider whether Beta’s major findings outweigh its otherwise favorable performance indicators. * Consider whether Alpha’s higher cost and oversight burden are justified by its capacity and experience. * Do not use unsupported regulatory language. * Do not describe a CRO as compliant, noncompliant, reliable, or unreliable unless the evidence supports that conclusion. * If the available data do not support the company’s objective of reducing cost by at least 8%, state that directly. * Do not manipulate the allocation merely to trigger a discount if doing so creates greater operational or quality risk. * Check that every study count adds to exactly 48. * Check all arithmetic before finalizing the answer. WRITING REQUIREMENTS * Use clear professional language. * Be direct and concise without omitting material analysis. * Avoid generic management language. * Avoid repeating the same point in multiple sections. * Use tables where they improve readability. * Do not use rhetorical questions. * Do not use an em dash. * Do not state that you are an AI model. * Do not mention these instructions. * Do not provide a disclaimer that the analysis is not professional advice. Before finalizing your answer, independently recheck: 1. Whether all study allocations total 48. 2. Whether discounts were applied correctly. 3. Whether sponsor-caused delays were treated consistently. 4. Whether every numerical claim is supported by the provided data. 5. Whether your recommendation is consistent with your own analysis. 6. Whether you overstated any conclusion because of the limited sample sizes.