MNI INTERVIEW: UK Soft, Hard Data Gap Hard To Pin Down - CBI

By Jamie Satchithanantham

     LONDON (MNI) - The UK manufacturing sector continues to see a disparity

between hard and soft output data, but the Confederation of British Industry are

yet to find a definitive reason why their Industrial Trends Survey continues to

diverge from official manufacturing data, Anna Leach, who heads up Economic

Intelligence at the UK's leading business lobby, told Market News in an

exclusive interview.

     Leach said that analysis into why a disparity had emerged between the two

datasets was ongoing, but identifying what was behind it was "not especially

clear".

     "Periods of divergence aren't especially unusual between the hard and soft

data such as ours ... One thing I have observed is that the nature of the

relationship between the two, though remaining close, has changed since 2012 and

we're still looking at why that might be," said Leach.

     The CBI's Industrial Trends Survey (ITS), along with other soft data

surveys like the IHS Markit Manufacturing PMI, have reported business conditions

holding up well since the UK's decision to leave the European Union last July

and more notably throughout 2017.

     In contrast, the official manufacturing data, procured by the Office of

National Statistics, though also proved resilient in the months immediately

after the referendum, showed a gradual deterioration as 2017 evolved.

     Logically, it would appear that either the official data was

underestimating business activity or the soft data overestimating it. To answer

this it is important to appreciate the differences between the surveys.

     Firstly, the sample sizes are vastly different. The ONS surveys roughly

5,000-6,000 businesses each month while the CBI and Markit surveys reach out to

roughly one-tenth this amount -- though still considered high.

     This, though, was dismissed as a possible explanation for the disparity

between the data by Leach.

     "We've done our usual checks to ensure we have no concerns about the

quality of our data. So we've not seen significant variation in the change in

size of our sample or in terms of the nature of the sample across the different

sectors," said Leach.

     Even with a smaller sample size, the CBI's data is weighted, with the

weights derived from those used by the ONS, which should alleviate any sample

bias.

     "We know there aren't any data errors in terms of the weighting

calculations we apply to our survey data. So, on the basis that it doesn't look

like anything in our control, that wouldn't give us cause for concern," she

added.

     DEPTH OD QUESTIONNAIRES COULD BE AN ISSUE

     The next potential reason would be how the survey questionnaires themselves

differ. The ONS, by virtue of the fact that businesses are obliged to

participate in its surveys, is permitted to ask more quantitative questions that

otherwise would put off survey respondents. By requesting the percentage change

in output growth over the month the ONS is thus able to capture depth of any

changes, something that alludes the soft data surveys.

     Surveys executed by the CBI, Markit and the like do not benefit from

compulsory participation and thus have to make the questions quick and easy to

answer as to encourage participation.

     These surveys typically ask whether output/new orders/etc. were higher or

lower or stable over the past month. So, while it can cover breadth, or

direction, of a change it cannot capture depth (how large the change was) in the

same way the ONS can. This is the classic argument often made by those who

discredit soft data surveys. 

     "That is a factor, other than differences in the approach used to

seasonally adjust the data, that could explain the divergence...it is possible

that we have a large number of companies saying that output has gone up when

output has only gone up a small amount," said Leach.

     According to Leach, the underlying assumption the CBI has made is that the

way the data is constructed implicitly assumes that what people assess as 'up'

or 'down' is stable over time and that there isn't significant variation in what

people define as 'up' or 'down'.

     To monitor this the CBI does run regular answering practices surveys (APSs)

which seek to understand what's behind respondent answers. Leach, however, did

note that the 2013 ITS APS did pick up some evidence that participant judgements

as to what changes in variables came under the heading 'same' had shifted

relative to the 2008 APS results. 

     For example, in 2008 45% of respondents judged a 2-4% change in a variable

small enough to fall under the 'same' option on the questionnaire when asked

whether that variable had increased, decreased of stayed the same over the

month. This fell to just above 30% in 2013. 

     Similarly, in 2008 around 12% thought a 4%-8% change in a variable was also

negligible enough to qualify selecting the 'same' option in 2008; in 2013 20%

thought this was acceptable.   

     These results, said Leach, provide some evidence that participant

approaches to answering survey questions may in fact move over time.

     That said, according to Leach the CBI does not collect sufficient data to

fully test the extent to which this may potentially explain some or all of the

current divergence with official data.

     Another factor they have examined is to what extent do the questions they

ask in their survey paint a coherent picture of what is going on in the sector.

     "Again, for the ITS there aren't any significant causes for concern," said

Leach, pointing to the fact that the survey's data on increased export orders

and higher unit input costs were both running with the Brexit-induced sterling

depreciation narrative.

     "It all fits," she said.

     Leach suggested the discrepancy could be down to the revision profile of

the ONS data, citing the previous revisions that have tended to bring the two

data series closer together.

     "There's nothing that would imply a significant degree of issue with our

data," said Leach.

     "The quarterly ITS has been running since the 50s and it's a pretty good

indicator of revised estimates of official data, which points to volatility in

earlier estimates of official data which is fairly well known," she said.

     Leach made reference to the most recent set of revisions made to the ONS'

data as part of the annual publication of the Blue Book National Accounts.

     Before these revisions, July was the first month where month-over-month

industrial production expanded for two consecutive months in 2017 and the first

where manufacturing output had expanded in 2017.

     Post revisions, August's IOP outturn was the fifth consecutive monthly rise

in production in 2017 and the third month of expansion in manufacturing during

the year.

     UPWARD REVISIONS TO ONS DATA COULD CUT DISPARITY

     According to Leach, these upward revisions "very slightly" reduced the

degree of divergence between ONS and CBI data but felt there was still a

"persistent divergence" which they had not been able to find a particular

explanation for.

     While being unable to explain the data gap, describing it as "frustrating",

Leach said her hunch was that the official data were underestimating the true

level of business activity in the sector.

     She added that conversations between the CBI and competitor IHS Markit had

taken place, as they had both experienced the same issue with their data.

     "It's something we'll keep a close eye on because we would like to be able

to explain what is driving the divergence but at the moment we think that maybe

conditions in the manufacturing sector are better than what the official data is

implying," said Leach.

--MNI London Bureau; +44 203-586-2226; email: jamie.satchithanantham@marketnews.com

[TOPICS: M$B$$$,M$E$$$,MX$$$$]