As part of a special issue on advances in art crime research, open-access journal Arts has published my study of metal detecting for cultural objects until ‘there is nothing left’: the potential and limits of digital data, netnographic data and market data for analysis.
I would particularly like to thank the peer-reviewers for enduring and Arts for publishing an exceptionally long piece, which presents the arguments over the methods, the raw data for the methods and some tests of the methods in one place. So, now, everything is available for reuse and reassessment.
My article reviews two methods of open-source research: “restricted quantitative analysis” of online forums; and “extensive quantitative analysis” of ‘online forums,… social networks, smartphone apps and offline associations, plus other empirical indicators…, such as statistics on detectorist reporters of cultural objects, owners of detectors and sales of detectors’. (These methods also allow qualitative analysis, but quantitative analysis is more problematic.) Then, it presents and tests methods for collecting and interpreting data on metal-detecting.
the largest online community is probably smaller than the total offline population
Although my article highlights that automatically-generated data may be unrepresentatively high, due to interference by activists and incidental influence by cybercriminals, it identifies a wide range of causes for unrepresentatively low levels of online social organisation, including the most basic ones.
Realistically, ‘significant numbers of online detectorists only participate in smaller online communities and do not participate in the largest online community’; ‘significant numbers of active detectorists only participate in offline activity and do not participate in any online community. So, ‘the membership of the largest online community is likely to represent a minimum estimate of the detecting population’.
It should also be remembered that, in general, ‘online community members who lurk invisibly appear to outnumber members who post visibly’. So, ‘a lack of visible online activity by online community members is not evidence of a lack of online activity, let alone a lack of offline activity’.
the measurable metal-detector market is significantly smaller than the total metal-detector market
It is difficult to account for the market in toy metal-detectors, the market in used metal-detectors (also known as second-hand detectors) and the manufacture of home-made, hand-built metal-detectors. Based on data from a study by Paul Barford, toy metal-detectors constitute an insignificant fraction of the market for new metal-detectors and the overall market in new and used metal-detectors is significantly larger than the visible market in new metal-detectors. This evidence offers ‘some reassurance that market data may provide a secure (under)estimate of detector consumption’.
in an affluent Western country, the average metal-detectorist may consume 0.32 metal-detectors per year
This article finally publishes the raw data for methods of estimating the metal-detecting population from the metal-detector market, which were initially discussed in a proof-of-concept study on estimation of the scale and intensity of metal detecting and the quantity of metal-detected cultural goods.
One significantly-sized meta-analysis, of polls and surveys on average ownership of metal-detectors, established the ownership (and so consumption) of at least 2,322 metal-detectors by at least 731 metal-detectorists, or 3.18 metal-detectors per metal-detectorist.
Another insignificantly-sized meta-analysis, of polls and surveys on average consumption of metal-detectors, suggested the consumption of 167 metal-detectors by 25 metal-detectorists over 530 person-years of metal-detecting, or 6.68 metal-detectors per metal-detectorist per 21.2 person-years of metal-detecting, or 0.32 metal-detectors per metal-detectorist per year.
there are perhaps 312,500 metal-detectorists in the United States
In the United States, the identifiable automatically-generated data suggest the existence of at least 110,578 metal-detectorists (around 1 detectorist in 2,922 people). However, the available market data suggest something else. First, the article corrects a miscalculation in the proof-of-concept study, which had suggested the existence of 160,000, instead of 1,562,500. Yet those numbers were derived from data during a peak in the market. So, then, the paper expands and refines the available market data, which ultimately suggest the existence of perhaps 312,500 (around 1 detectorist in 920 people).
there are perhaps 31,250 metal-detectorists in France
In France, the identifiable automatically-generated data suggest the existence of at least 20,385 metal-detectorists (around 1 detectorist in 3,282 people). However, the available market data suggest the existence of perhaps 31,250 (around 1 detectorist in 2,141 people).
there are at least 2,858 metal-detectorists in Hungary
In Hungary, the available market data suggest the existence of perhaps 1,258 metal-detectorists (around 1 detectorist in 7,967 people). However, the identifiable automatically-generated data suggest the existence of at least 2,858 (around 1 detectorist in 3,435 people).
detectorists don’t only use online forums, so researchers shouldn’t only analyse online forums
‘Detectorists not only use different structures for online social organisation, but themselves identify the use of different structures in different territories…. So, unless they are studying particular communities, it is illogical for researchers to analyse only online forums…. even extensive analysis of trends in national social organisation may underestimate activity in territories with trends towards transnational social organisation’, such as countries in South-East Asia.
‘exclusively forum-derived data are inadequate’
‘It has been asserted that the sizes of countries’ largest online forums “represent roughly the same percentage of the overall number of metal detectorists in all transnationally compared [i.e., comparable] countries”… and that it would be “ridiculous” to assume the “exceedingly unlikely” alternative…. Contrarily, the proportions of communities who participate in any and all venues do vary.’
Comparing different sources of automatically-generated data, for instance, ‘for the United States, the largest online forum is currently around 1080.48 percent of the size of the largest social network…; for France, it is 440.56 percent…; for Poland, it is 163.15 percent…; for Russia, it is 111.17 percent…. Yet, for Ukraine, it is 93.92 percent…; for the United Kingdom, it is 70.31 percent…; for Hungary, it is 31.61 percent…’. Evidently, neither online forums nor social networks have the same appeal in different territories.
Comparing automatically-generated data with market data, in the United States, ‘the largest online community represents perhaps 35.38 percent of the detecting population’; in France, ‘the largest online community represents perhaps 65.23 percent of the detecting population’. These examples are ‘yet more evidence that the largest online communities – which, in the cases of the United States and France, are the largest online forums – do not represent a constant fraction of detecting populations.’
Meanwhile, ‘the market data analysis, which was derived from data from affluent societies in North America and Europe, only identified perhaps 44.02 percent of the detecting population’ in less affluent Hungary. ‘This demonstrates that the measure of market activity is “culture-bound”. If it is not based on comparable market data, market data analysis may severely misrepresent the scale of metal-detecting.’
by its own logic, restricted quantitative analysis is unsound
According to advocates of restricted quantitative analysis, its results ‘”can” theoretically “be accepted as sound and conclusive” if the number of online forum members is “reasonably representative of the number of metal detectorists”‘. Since the number of online forum members is not reasonably representative of the number of metal-detectorists, since online forum membership represents a different proportion of the total metal-detecting population in each territory, ‘the results of restricted quantitative analysis must be rejected as unsound’. ‘Neither method is conclusive.’
So, ‘it is necessary to make tentative, “least worst” estimates of actual numbers, which are derived from an extensive range of sources. Equally, it is necessary to recognise that those least worst estimates that are derived from automatically-generated data are only minimum numbers, not constant fractions of actual numbers.’
metal-detectorists recognise that they are “beating sites to death”
This study demonstrates that a lot of convincing evidence is there to be found, if it is sought. We have the capacity to refine our understanding of the problem and so our response to it. ‘In this sample, the results appear to revise established estimates and corroborate the detecting community’s own perception that they are “beating these sites to death”.’
abstract
This methodological study assesses the potential for automatically generated data, netnographic data and market data on metal-detecting to advance cultural property criminology. The method comprises the analysis of open sources that have been identified through multilingual searches of Google Scholar, Google Web and Facebook.
Results show significant differences between digital data and market data. These demonstrate the limits of restricted quantitative analysis of online forums and the limits of extrapolation of market data with “culture-bound” measures. Regarding the validity of potential quantitative methods, social networks as well as online forums are used differently in different territories. Restricted quantitative analysis, and its foundational assumption of a constant relationship between the size of the largest online forum and the size of the metal-detecting population, are unsound.
It is necessary to conduct extensive quantitative analysis, then to make tentative “least worst” estimates. As demonstrated in the sample territories, extensive analyses may provide empirical data, which revise established estimates. In this sample, they corroborate the detecting community’s own perception that they are ‘beat[ing these sites] to death’.
citation
Hardy, S A. 2018: “Metal detecting for cultural objects until ‘there is nothing left’: The potential and limits of digital data, netnographic data and market data for analysis”. Arts, Volume 7, Number 3. Available at: https://doi.org/10.3390/arts7030040
conflict antiquities 