A new study finds ChatGPT talks about the same brand far more harshly when the question is skeptical, and repeat runs of one prompt agreed on its tone only 44% of the time.

Method in one clause: 180 matched prompts (60 positive, 60 neutral, 60 negative) about three brand pairs (Samsung and Sony TVs, Hers and Start Willow GLP-1 telehealth, Delta SkyMiles and United MileagePlus), each run five times in ChatGPT for 900 answers, tone scored from -100 to +100 without the scorer seeing the prompt, runs averaged per prompt. Older than our usual window (4 November) but new to us.

The figures: average net tone was -36 for negative prompts, +20 for neutral and +46 for positive, so negative wording moved tone 56 points and positive wording 26. The correlation between prompt tone and answer tone was 0.66 (Spearman), and 0.62 to 0.78 within each category. The author says the gap survives allowing for the ceiling effect. On trust questions (reliability, refunds, privacy, safety), positive wording barely helped, while negative wording still hurt. In reversed comparison prompts, United MileagePlus swung from +91 to -93. Stability: only 44% of prompts got the same tone label in all five runs, net tone varied an average of 27 points between runs, and 96% had a majority verdict (three of five agreeing). Even neutral questions came back mildly positive (+20 on average).

Limits: SE Ranking sells a tracker; three consumer categories with big brands, no services or coaching; ChatGPT only; the date and model of the runs are not stated on the page.