The Sheffield Alcohol Research Group claim the interim evaluation was flawed.
In analysis published by the British Medical Journal the group has called for the report to be withdrawn and targets revised.
Dr John Holmes, the author of the report, said: “The Department of Health and their partners in the Responsibility Deal have promoted the Billion Unit pledge as an example of what can be achieved by government working in partnership with the alcohol industry.
“However, there are flaws in the Department of Health’s interim evaluation which are sufficiently serious that we believe the report should be withdrawn along with claims that the target of removing one billion units from the market has been met.”
Simplistic
The research team argue that a closer look at the analyses and data that underpin the headline figure raises questions about how much of the recent changes in alcohol consumption are truly attributable to the pledge.
They believe the data used in the analysis “may not be fit for purpose, that the report makes simplistic assumptions about consumer responses to the pledge, and takes insufficient notice of confounding factors.”
For example, they suggest a change to the way HM Revenue and Customs (HMRC) recorded beer data and the introduction of lower taxes on lower strength beers may account for some of the estimated effect of the pledge.
They say an assumption is also made in the analysis that the pledge will lead people to simply drink the same amount of beer and wine but at a lower strength. The researchers argue this is simplistic and means the analysis will give misleading results.
Alternatives
They acknowledge that their critique “does not imply the billion unit pledge is bad for public health” but they claim that a lack of appropriate data means a rigorous evaluation of whether the pledge has been met may not be possible.
They recommend that the Department of Health “withdraws the 2014 interim report, requests stakeholders not to cite its conclusions, and reviews the evaluation approach.”
They also recommend the billion unit target “is abandoned in favour of measurable alternatives" - for example reporting on what products have been introduced or altered in strength, and supplying sales data for these products.
Positive
In response, Brigid Simmonds, chief executive of the British Beer and Pub Association, said: "We are extremely proud of our track record under the billion unit pledge.
"As the official Government statistics show, beer has contributed the overwhelming majority of alcohol units saved under the pledge, through both reducing the strength of existing brands, and also through innovation with new, lower-strength products.
"We support the methodology used by the DOH, but of course, many factors will influence changes in consumer behaviour, and statisticians have to take a view, when introducing controls for these factors.
"This in no way undermines the fact that the figures are hugely positive, and show that partnerships between industry and Government can make an important contribution in tackling alcohol-related harms. This is positive action that all should support; and action we hope will continue after the General Election."
A spokesperson for the Portman Group said: “This is just another poorly-evidenced article written by a research group with clear campaigning objectives, and published by the BMJ – a journal with a growing reputation for inaccuracy and insinuation.
"By working in partnership with government, the drinks industry has reduced the strength of well-known brands, introduced new lower-strength products and limited the number of units of alcohol in single serve cans. This kind of innovation is a win for consumer choice, a win for public health and has resulted in 1.3 billion units being removed from the UK market – a fact verified by the Department of Health.
"We have to start questioning the motives of those who are so desperate to challenge official government statistics and undermine the vital work being done to improve public health.”