CardioBrief
One-stop source for cardiology news and links
More Blasts Of Concern Over ROCKET-AF
September 28, 2016 by Larry Husten 1 Comment
The controversy over the big Xarelto trial does not appear to be going away.
A new investigation published in the BMJ raises more troubling questions about the ROCKET-AF trial, which compared the novel oral anticoagulant rivaroxaban (Xarelto, Johnson & Johnson) to warfarin in patients with atrial fibrillation. The controversy about the trial first unfolded last fall when it became known that the portable Alere devices used to monitor and calibrate warfarin usage in the trial were seriously defective. Since then investigators, industry, regulators, and critics have been trying to assess whether the defective devices compromised the findings of the trial.
The new development in the story is that shortly after the trial started J&J employees became aware that questions were being raised about the monitoring device in a small number of cases. The company launched a program to investigate these concerns but, according to BMJ associate editor Deborah Cohen, failed to inform either regulators or the trial's data and safety monitoring board (DSMB) about the findings of the program.
The BMJ reports that 149 samples were sent to the company by trial investigators who had questions about the accuracy and reliability of the INR readings. 71 of the samples were from patients randomized to warfarin. (Fake INR readings were generated for patients who were randomized to rivaroxaban.) According to the company there were "16 instances where the value from the point of care device and lab were… inconsistent." Thus in this small, nonrepresentative sample nearly one quarter of the portable readings used in the trial did not match standard laboratory readings.
The BMJ quotes a member of the trial's DSMB, Peter Rothwell (Oxford University), who said he had "no memory of the board being told" about the findings. "Clearly, if the sponsor of the trial had concerns about the validity of the point-of-care testing of INR that would have been important for the board to be made aware of [them]," he told the BMJ. Neither did the company inform either the FDA or the EMA about the findings.
One ROCKET-AF investigator, Markku Kaste (Helskini University General Hospital) agreed that the findings should have been shared. "I now have some doubts about the validity of ROCKET AF trial. It possibly skews the data in favour of rivaroxaban," he said.
Defending ROCKET-AF
J&J responded that the company behaved properly because there was no good reason for it to report this data. I spoke with Pete DiBattiste, the head of development for cardiovascular drugs Janssen (the pharmaceutical arm of J&J), who told me that the company "took great care and are highly confident that we submitted all relevant information not only to the FDA but to all health authorities around the globe." He pointed out that the 149 samples submitted by investigators to the program represented less than one-twentieth of 1% of the approximately 366,000 INR measurements obtained in the trial.
The small number of samples sent by investigators "is itself a sign of reassurance," said DiBattiste. The program "was put in place as a safety check" but "ended up yielding no meaningful issue" and so there was no reason to report the findings to either the DSMB or regulators, he said. He also pointed out that a certain amount of discordance would be expected, and that the FDA standard for approval of devices does not require perfect accordance.
J&J has also defended ROCKET-AF on the basis of analyses performed by Janssen and its European partner, Bayer, as well as separate analyses performed by the ROCKET-AF investigators. But these analyses have also been subject to criticism. In her article Cohen cites numerous questions that have been raised about these analyses.
"We need to find ways to reduce uncertainty and increase clarity about the balance of benefit to harm," write Kamal Mahtani and Carl Heneghan in an accompanying BMJ editorial. Ideally they would like to see independent trials to replicate the results but they acknowledge that "this may take several years. In the mean time, making the data available for independent scrutiny should be a mandatory regulatory requirement, particularly when there are questions about trial rigour." For now, however, "patients and clinicians must… live with the uncertainty left by the evidence currently available."
In a second accompanying commentary, Vinay Rathi, Harlan Krumholz, and Joseph Ross discuss the larger regulatory issues involving diagnostic devices raised by this case. They note that diagnostic devices "rarely require prospective clinical studies for clearance" and that devices that have been recalled can be used as predicates for the approval of new devices. Even the FDA acknowledges that "there are few performance standards… based on clearly defined scientific parameters."
Perspective From Sanjay Kaul
Sanjay Kaul (Cedars Sinai) was a member of the FDA panel that reviewed ROCKET-AF He told me that he thinks Janssen should have disclosed the data to either the DSMB or the regulators. "It was a mistake that continues to fuel the controversy and perpetuate the perception of malfeasance." Kaul does not find strong evidence in the available data suggesting that the ROCKET-AF results are invalid, but he does support "an reassessment of benefit-risk by independent academics" to perhaps "dispel any residual doubt or adjudicate remaining uncertainties."
Here is Kaul's full statement:
"The saga continues!
"I find it unacceptable that Janssen did not disclose the results of the Covance recheck program to the DSMB or the regulators. It was a mistake that continues to fuel the controversy and perpetuate the perception of malfeasance.
"I am all for data transparency and independent replication as suggested by the editorialists. However, the key question here is whether there is sufficient evidence that the benefit-risk balance of rivaroxaban was materially impacted by the faulty INR readings to revoke the approved indication. The EMA conducted an 'independent' investigation and concluded the benefit-risk balance remained unchanged. The FDA is also conducting its own investigation and so far has not made any major pronouncements. I don't want to second guess the FDA, but if there was an obvious 'fatal flaw' to render the trial data invalid, they would have acted sooner in the interest of public safety.
"Here is my own personal perspective.
Of the 767 values where lab INR>4, 172 (22%) device INR values were between 2 and 3 and 47 (6%) device INR values were less than 2. A device INR of <2 (a 2-INR category discordance) is likely to lead to dose adjustment (escalation). It is important to keep in mind that dose adjustments are seldom based on single INR values. Please note that the difference in bleeding in those with vs without discrepancy is 1.5/100 patient years to 2.0/100 patient years. This is a marginal difference in a relatively small subset (6% of cohort) which is not likely to result in a MATERIAL difference in the benefit-risk balance. Even if this discrepancy was observed in 100% of the cohort (worst-case scenario), I am not convinced this would result in an undesirable benefit-risk balance. It is reassuring to learn that in data sets external to ROCKET-AF trial (where the defective INR device was not utilized), warfarin-related bleeding rates were consistent with those observed in ROCKET-AF.
"What compounds the issue is that dose adjustment resulting from a clinically meaningful discrepancy in INR values would have expected to result in lower efficacy event rates (stroke and systemic embolism) and higher safety event rates (bleeding) with warfarin. While observed bleeding rates were aligned with the expected rates, observed stroke rates were the opposite of expected (higher rates). In addition, why would rivaroxaban (fixed doses were used) bleeding outcomes also be affected in a directionally similar manner as with warfarin by INR discrepancy but not stroke? These data appear to suggest that observed differences in warfarin event rates cannot be primarily attributed to the adjustments made in warfarin dose based on discrepancy-related device INRs.
"Bottom line, short of conducting a new trial, perhaps, an reassessment of benefit-risk by independent academics might be able to dispel any residual doubt or adjudicate remaining uncertainties amongst the skeptics."
Previous coverage:
Alere Withdraws Anticoagulation Monitoring System At Heart Of ROCKET AF Controversy (July 13, 2016)
New Studies Seek To Allay Concerns Over The ROCKET-AF Trial (February 3, 2016)
ROCKET-AF Investigators Seek To Calm Concerns About Trial Reliability (December 13, 2015)
New Questions Raised About ROCKET AF, Pivotal Xarelto Trial Chaired By Califf (November 30, 2015)
Filed Under: People, Places & Events, Policy & Ethics Tagged With: Alere, BMJ, INR, rivaroxaban, ROCKET-AF, Xarelto
I have no experience with trials like this, but I do have a lot of experience of laboratory science. If you find that your instruments are dud it's usually a fool's errand to try to rescue useable data from the experiment. Scrap the lot and start again usually proves best.
Carol Vassar, MD on Skeptics and the Pandemic
Jeffrrey S. Rose MD on Skeptics and the Pandemic
Louis Krut on Skeptics and the Pandemic
Joseph on Heart Failure Death Statistics: Don't believe what you read on the internet
PAD vs. Neuropathy on FDA Approves Zilver PTX Drug-Eluting Stent For Peripheral Arterial Disease
Archives Select Month September 2021 (1) January 2021 (1) December 2020 (1) October 2020 (1) September 2020 (1) August 2020 (2) July 2020 (1) June 2020 (1) November 2019 (1) March 2019 (1) December 2018 (1) October 2018 (1) September 2018 (2) August 2018 (2) July 2018 (4) June 2018 (2) May 2018 (5) March 2018 (4) February 2018 (5) January 2018 (1) December 2017 (7) November 2017 (12) October 2017 (10) September 2017 (9) August 2017 (12) July 2017 (11) June 2017 (11) May 2017 (11) April 2017 (9) March 2017 (12) February 2017 (6) January 2017 (3) December 2016 (7) November 2016 (18) October 2016 (14) September 2016 (12) August 2016 (15) July 2016 (10) June 2016 (13) May 2016 (15) April 2016 (17) March 2016 (16) February 2016 (13) January 2016 (13) December 2015 (15) November 2015 (21) October 2015 (25) September 2015 (28) August 2015 (15) July 2015 (4) June 2015 (5) May 2015 (8) April 2015 (16) March 2015 (19) February 2015 (13) January 2015 (20) December 2014 (11) November 2014 (17) October 2014 (17) September 2014 (20) August 2014 (21) July 2014 (18) June 2014 (18) May 2014 (20) April 2014 (16) March 2014 (30) February 2014 (21) January 2014 (27) December 2013 (18) November 2013 (24) October 2013 (28) September 2013 (23) August 2013 (13) July 2013 (18) June 2013 (20) May 2013 (29) April 2013 (24) March 2013 (31) February 2013 (25) January 2013 (32) December 2012 (24) November 2012 (32) October 2012 (27) September 2012 (25) August 2012 (34) July 2012 (23) June 2012 (25) May 2012 (30) April 2012 (26) March 2012 (32) February 2012 (25) January 2012 (33) December 2011 (30) November 2011 (37) October 2011 (26) September 2011 (29) August 2011 (30) July 2011 (29) June 2011 (31) May 2011 (42) April 2011 (36) March 2011 (39) February 2011 (30) January 2011 (30) December 2010 (28) November 2010 (34) October 2010 (30) September 2010 (35) August 2010 (32) July 2010 (35) June 2010 (35) May 2010 (22) April 2010 (24) March 2010 (43) February 2010 (23) January 2010 (33) December 2009 (38) November 2009 (47) October 2009 (36) September 2009 (41) August 2009 (39) July 2009 (41) June 2009 (45) May 2009 (38) April 2009 (44) March 2009 (72) February 2009 (65) January 2009 (44)
Note to Readers: A Time For Change
Click here to read about important new changes to CardioBrief.
Follow Cardiobrief on Twitter
Developed By: SKSDEV
CardioBrief Classics
Coca-Cola, The Olympic Torch, And The American College Of Cardiology
Stem Cell Therapy Company Hypes Preliminary Results
Good Science/Bad Science: Contrasting Papers On Dietary Compositon In JAMA And BMJ
Subway Meals Get American Heart Association Endorsement
This Week In Medicine: Stop Exercising and Eat Chocolate!
You Know Nothing, Dr. Snow: Why Medicine Can't Be More Like Facebook
Politics and Transcatheter Aortic Valve Replacement
Half the News That's Fit To Print: NY Times On ECG Screening For Student Athletes
New Perspective on the Dutch Cardiovascular Research Scandal
Mark Midei Can't Get a Job Taking Blood Pressure At A Walmart
And the 2012 Award For the Most Dumbass Drug Promotion Goes To…
Guest Post: More Lessons From the Riata ICD Lead Recall
Japanese Researcher With Harvard Connections Retracts 3 Articles in AHA Journals
Lancet Editor Richard Horton Tweets Dark View of Contemporary Medicine
Resveratrol and Fraud
Director of UConn CV Research Center Accused of Scientific Misconduct
2011 in Review: Rivaroxaban, Sapien, Mark Midei, Conflicts of Interest, and Much More
Nissen and Topol Clash Over Genetics and Personalized Medicine
Ray of Light for the Physician Payment Sunshine Act
Letter From Jail: A Cautionary Tale
One More Thought About Lipitor
Experts Clash Over Interpretation and Value of AIM-HIGH
Controversy Over Cholesterol Testing for Children
CardioBrief Poll: Should the ACC Train Industry Reps?
Daily Show Asks Why the AHA Killed a Bill Encouraging Healthy Behavior
National Lipid Association Expert Panel Has Many Deep Ties To Industry
Industry & Medicine: It's Complicated
Easy Come, Easy Go? ESC To Review Dronedarone's Role in AF Guidelines
The Pen Is Mightier Than the Scold
CNN Documentary Promises Screening and Vegan Diet Can Eliminate Heart Attacks
CNN, ABC, and NBC Dumb Down the News About CV Screening
US Senate Unanimously Endorses Campaign Against Atrial Fibrillation
Less May Be More, But Stents Are Neither Good Nor Bad
The Voice of the Patient: Time To Bring Out the Muzzle?
New Concerns Raised About Withdrawn Archives Meditation Paper
Archives Decides At Last Minute Not to Publish a Scheduled Paper
Part 2: The National Lipid Association And Industry
Part 1: The National Lipid Association and the FH Guidelines
Understanding Risk: Tricky Business
ACC CEO Jack Lewin Provides The Argument Against Industry Money
The ACC Counts Down To Launch Of New Satellites
Scientific Fraud Found In Wayne Alexander's Research Group At Emory
$50K: The Price For A VIP Dinner For 2 With The Cleveland Clinic CEO
Evidence: The Weak Link Of Evidence-Based Medicine
Taking Medical Advice From Peggy Fleming?
The ACC Wants You To Know That Nothing Is Rotten In The State Of Maryland
An Embargo Becomes A Public Relations Tool
PROTECT II: Who Will Protect Us From The Protectors?
GRAVITAS Published: Who Gets To Weigh In On the Results?
WebMD: the 800 Pound Gorilla in the Room
Some Things Don't Go Better With Coke
ACC and HRS Downplay Data From Their Own Registry
Dangling Doctors In Front of Journalists
Spinning Out of Control: Another Look at the PROTECT II Press Release
Questions Raised About Compensation and Conflicts at Duke
SHARP: A Tale of Two Press Releases
Will This Be the HDL Decade? REVEAL Revealed, Mixed Results for Apo-A1
CardioExchange at the AHA: A New Voice Emerges
Top New York City Interventional Cardiologists Now Making $3 Million a Year
The Plot Thickens in the Case of the Exploding ICD
Red light or green light for AAA screening?
Blue light special: AAA screening at Kmart in the disease-mongering aisle
The rise of the ESC: is the ESC anti-American or pro-industry?
Dronedarone and ticagrelor speed into ESC guidelines
The debate over CME and industry influence continues
Avandia aftermath: who are the winners and losers?
Debating medical promotion and education
Why is there a red dress on diet Coke cans?
SHAPE shifting: transforming guidelines into a business venture
Why was an Abbott marketing study published in the American Journal of Cardiology?
At the Dallas valve meeting even the faculty is for sale
A missing piece of the Multaq puzzle finally appears: DIONYSOS published to little fanfare
EHJ editors rebuffed GSK efforts to suppress Nissen editorial on rosiglitazone
Why we shouldn't kill all the lawyers just yet
Sunday at the AHA: ARBITER's wild roller coaster ride in Orlando
Cardiology $$$: Mt Sinai's Samin Sharma tops Columbia's Moses
Residual Risk Reduction Initiative: promotion or education?
Editorial: the incompetence of the FDA
Editorial: SHAPE is skating on thin ice
Extreme cardiology culture: Califf, Harrington, Topol & Teirstein
October thought experiment: suppose the World Series were covered like the Nobel Prize
Live AF ablation broadcast on Today Show: what's wrong with this picture?
More Multaq: Prystowsky whack-a-mole, ACC responds, Sanofi overtures to EPs
Pieces of a puzzle: Multaq, Sanofi, ACC, HRS, Prystowsky, AF Guidelines
"I'd rather read Larry Husten on clinical trials than the constipated editorials in peer review journals." --Saurabh Jha (The Health Care Blog)